2024-09-16T22:05:26.143 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T22:05:26.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T22:05:26.326 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596 branch: wip-dis-testing description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: idryomov@gmail.com first_in_suite: false job_id: '7907596' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2961 sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 sleep_before_teardown: 0 subset: 66/100 suite: rbd suite_branch: wip-dis-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGORqTOgq1LQ4dFy+zGK+rqmgXrEzhhUuBDjzVadRA4VnDZbZEf1OYfiZXZDBbq0BpZEYhzJ5Qf6DOLnZYYgfs= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaY0tqkGzJZHIAvKvbsOPTEZPEwMoT2OpTXZ/fSCljvk5pHpnCHkLPcf95R4RXK41U+0g9CSqwE7p0PvAFbkyM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_19:45:14 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T22:05:26.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa; will attempt to use it 2024-09-16T22:05:26.327 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa/tasks 2024-09-16T22:05:26.327 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T22:05:26.328 INFO:teuthology.task.internal:Checking packages... 2024-09-16T22:05:26.350 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6e8474a02f2b86353394076ab80c6f50ea28e361' 2024-09-16T22:05:26.350 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T22:05:26.350 INFO:teuthology.packaging:ref: None 2024-09-16T22:05:26.350 INFO:teuthology.packaging:tag: None 2024-09-16T22:05:26.350 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-16T22:05:26.350 INFO:teuthology.packaging:sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T22:05:26.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-dis-testing 2024-09-16T22:05:26.510 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4883.g6e8474a0 2024-09-16T22:05:26.512 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T22:05:26.517 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T22:05:26.518 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T22:05:26.544 INFO:teuthology.task.internal:Saving configuration 2024-09-16T22:05:26.556 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T22:05:26.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T22:05:26.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 22:00:31.105970', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGORqTOgq1LQ4dFy+zGK+rqmgXrEzhhUuBDjzVadRA4VnDZbZEf1OYfiZXZDBbq0BpZEYhzJ5Qf6DOLnZYYgfs='} 2024-09-16T22:05:26.605 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 22:00:31.107777', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaY0tqkGzJZHIAvKvbsOPTEZPEwMoT2OpTXZ/fSCljvk5pHpnCHkLPcf95R4RXK41U+0g9CSqwE7p0PvAFbkyM='} 2024-09-16T22:05:26.606 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T22:05:26.611 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-16T22:05:26.612 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-16T22:05:26.612 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T22:05:26.682 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc04f5cbac0>, signals=[15]) 2024-09-16T22:05:26.682 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T22:05:26.688 INFO:teuthology.task.internal:Opening connections... 2024-09-16T22:05:26.688 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:05:26.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:05:26.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:05:26.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:05:26.841 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T22:05:26.847 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-16T22:05:26.864 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-16T22:05:26.865 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-16T22:05:26.922 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T22:05:26.923 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-16T22:05:26.923 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T22:05:26.923 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T22:05:26.923 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T22:05:26.923 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-16T22:05:26.943 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-16T22:05:26.960 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-16T22:05:26.961 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-16T22:05:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-16T22:05:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-16T22:05:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-16T22:05:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-16T22:05:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T22:05:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T22:05:27.018 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-16T22:05:27.043 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T22:05:27.053 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T22:05:27.079 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T22:05:27.079 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-16T22:05:27.082 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-16T22:05:27.097 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T22:05:27.102 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T22:05:27.103 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-16T22:05:27.139 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-16T22:05:27.216 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T22:05:27.267 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T22:05:27.282 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T22:05:27.283 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T22:05:27.283 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T22:05:27.284 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T22:05:27.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:05:27.301 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-16T22:05:27.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:05:27.302 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T22:05:27.358 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-16T22:05:27.359 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-16T22:05:27.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-16T22:05:27.360 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-16T22:05:27.360 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-16T22:05:27.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-16T22:05:30.917 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.3 MB 00:01 2024-09-16T22:05:31.396 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.3 MB 00:02 2024-09-16T22:05:32.743 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-16T22:05:33.838 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 15 MB/s | 20 MB 00:01 2024-09-16T22:05:38.243 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-16T22:05:39.910 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-16T22:05:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-16T22:05:40.781 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T22:05:41.848 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-16T22:05:42.736 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T22:05:46.528 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T22:05:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T22:05:48.509 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T22:05:48.509 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T22:05:48.604 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T22:05:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-16T22:05:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:05:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-16T22:05:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-16T22:05:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:05:49.021 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 126 kB/s | 36 kB 00:00 2024-09-16T22:05:50.005 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T22:05:50.006 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T22:05:50.094 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:05:50.103 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:05:50.103 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-16T22:05:50.103 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:05:50.103 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-16T22:05:50.104 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:05:50.105 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-16T22:05:50.105 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-16T22:05:50.105 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:05:50.159 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 199.223.246.113) 2024-09-16T22:05:50.163 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 199.223.246.113) 2024-09-16T22:05:50.163 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 199.223.246.113) 2024-09-16T22:05:50.205 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.205 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.206 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.278 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.286 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.293 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.359 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 154 kB/s | 36 kB 00:00 2024-09-16T22:05:50.380 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T22:05:50.397 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 199.223.246.113) 2024-09-16T22:05:50.809 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-16T22:05:51.359 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-16T22:05:52.676 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-16T22:05:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:05:52.679 INFO:teuthology.orchestra.run.smithi092.stdout:Total 31 MB/s | 79 MB 00:02 2024-09-16T22:05:53.358 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:05:53.702 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:05:53.703 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:05:54.649 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:05:54.649 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:05:55.329 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.4 MB/s | 16 MB 00:06 2024-09-16T22:05:57.081 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:05:57.687 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-16T22:05:57.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T22:05:57.826 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:06:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:06:00.714 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:06:03.186 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.0 MB/s | 29 MB 00:14 2024-09-16T22:06:03.187 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:06:03.187 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.4 MB/s | 79 MB 00:14 2024-09-16T22:06:03.896 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:06:04.277 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:06:04.277 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:06:05.269 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:06:05.270 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:06:05.567 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:06:05.614 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:06:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T22:06:08.623 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:06:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:06:11.501 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:06:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:06:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:48.896 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:48.936 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:49.347 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:49.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T22:06:49.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:06:49.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:06:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:06:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:06:50.175 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:06:50.501 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T22:06:50.579 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.579 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T22:06:50.579 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T22:06:50.579 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T22:06:50.580 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T22:06:50.580 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T22:06:50.580 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T22:06:50.580 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T22:06:50.639 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T22:06:50.639 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T22:06:50.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-16T22:06:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T22:06:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T22:06:50.682 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T22:06:50.682 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-16T22:06:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:06:50.682 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T22:06:50.683 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T22:06:50.683 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-508.el9.x86_64 2024-09-16T22:06:50.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T22:06:51.375 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-16T22:06:52.745 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T22:06:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-16T22:06:52.787 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-16T22:06:52.805 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.BOli58iOEB 2024-09-16T22:06:52.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BOli58iOEB 2024-09-16T22:06:52.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.BOli58iOEB 2024-09-16T22:06:53.017 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.BOli58iOEB is 6KB 2024-09-16T22:06:53.066 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.BOli58iOEB 2024-09-16T22:06:53.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T22:06:53.143 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T22:06:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T22:06:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T22:06:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T22:06:53.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T22:06:53.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-16T22:07:00.819 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:07:00.860 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T22:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T22:07:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T22:07:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T22:07:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:07:02.161 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:07:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:07:02.162 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:07:02.502 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T22:07:02.635 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.635 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T22:07:02.635 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T22:07:02.636 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T22:07:02.636 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T22:07:02.636 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T22:07:02.636 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T22:07:02.636 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T22:07:02.694 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T22:07:02.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T22:07:02.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-16T22:07:02.730 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-16T22:07:02.731 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-16T22:07:02.731 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-16T22:07:02.731 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-16T22:07:02.731 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-16T22:07:02.732 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T22:07:02.732 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-16T22:07:02.732 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-16T22:07:02.732 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-16T22:07:02.732 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-16T22:07:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:07:02.734 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T22:07:02.736 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T22:07:02.736 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-508.el9.x86_64 2024-09-16T22:07:02.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T22:07:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-16T22:07:04.620 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T22:07:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-16T22:07:04.643 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-16T22:07:04.658 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.OythmeLyO8 2024-09-16T22:07:04.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OythmeLyO8 2024-09-16T22:07:04.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.OythmeLyO8 2024-09-16T22:07:04.869 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.OythmeLyO8 is 6KB 2024-09-16T22:07:04.919 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.OythmeLyO8 2024-09-16T22:07:04.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T22:07:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T22:07:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T22:07:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T22:07:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T22:07:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T22:07:05.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-16T22:07:23.325 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T22:07:23.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:07:23.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:07:35.168 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T22:07:35.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:07:35.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:07:41.885 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-16T22:07:50.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:07:50.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:07:53.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T22:07:53.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-16T22:08:02.669 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:08:02.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:08:05.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:08:05.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:08:21.057 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T22:08:24.385 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-16T22:08:33.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:08:33.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:08:36.129 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T22:08:39.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:08:39.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:08:39.744 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-16T22:08:40.134 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:08:40.134 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-16T22:08:40.135 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-16T22:08:40.149 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-508.el9.x86_64 2024-09-16T22:08:40.149 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T22:08:40.150 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T22:08:40.150 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T22:08:41.150 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T22:08:41.151 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T22:08:41.260 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-16T22:08:41.280 DEBUG:teuthology.parallel:result is None 2024-09-16T22:08:51.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:08:51.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:08:51.544 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T22:08:51.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:08:51.889 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-16T22:08:51.890 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T22:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-508.el9.x86_64 2024-09-16T22:08:51.906 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T22:08:51.906 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T22:08:51.906 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T22:08:52.907 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T22:08:52.907 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T22:08:52.984 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-16T22:08:53.013 DEBUG:teuthology.parallel:result is None 2024-09-16T22:08:53.013 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T22:08:53.023 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T22:08:53.024 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T22:08:53.026 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T22:08:53.045 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T22:08:53.057 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T22:08:53.089 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T22:08:53.090 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T22:08:53.093 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T22:08:53.132 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T22:08:53.143 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T22:08:53.144 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T22:08:53.163 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T22:08:53.195 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T22:08:53.200 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T22:08:53.206 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T22:08:53.211 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T22:08:53.213 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T22:08:53.222 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T22:08:53.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T22:08:53.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T22:08:53.281 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T22:08:53.292 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T22:08:53.292 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T22:08:53.320 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T22:08:53.337 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T22:08:53.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T22:08:53.470 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T22:08:53.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T22:08:53.596 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:08:53.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T22:08:53.663 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T22:08:53.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T22:08:53.758 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T22:08:53.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T22:08:53.885 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:08:53.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T22:08:53.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-16T22:08:53.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-16T22:08:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T22:08:54.045 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T22:08:54.527 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T22:08:54.534 INFO:teuthology.task.internal:Starting timer... 2024-09-16T22:08:54.534 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T22:08:54.567 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T22:08:54.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-16T22:08:54.652 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-16T22:08:54.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-16T22:08:54.719 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-16T22:08:54.721 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T22:08:54.721 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-16T22:08:54.763 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-16T22:08:54.764 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-16T22:08:54.802 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-16T22:08:54.803 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive'} 2024-09-16T22:08:54.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T22:08:54.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:08:54.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T22:08:54.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:08:54.870 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T22:08:54.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-16T22:08:54.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-16T22:08:54.957 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T22:08:54.966 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T22:08:54.977 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T22:08:54.978 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T22:12:45.128 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-09-16T22:12:45.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:12:45.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:12:45.214 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T22:12:45.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T22:12:45.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:12:45.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T22:12:45.368 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-16T22:12:45.449 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-16T22:12:45.449 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T22:12:45.459 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T22:12:45.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T22:12:45.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T22:12:45.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T22:12:45.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T22:12:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T22:12:45.507 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T22:12:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T22:12:45.533 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-16T22:12:45.535 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T22:12:45.545 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-16T22:12:45.560 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T22:12:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-16T22:12:45.571 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-16T22:12:45.575 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T22:12:45.585 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T22:12:45.599 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T22:12:45.633 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.638 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.652 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.655 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:12:45.656 INFO:teuthology.run_tasks:Running task install... 2024-09-16T22:12:45.664 DEBUG:teuthology.task.install:project ceph 2024-09-16T22:12:45.664 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-16T22:12:45.664 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'} 2024-09-16T22:12:45.664 INFO:teuthology.task.install:Using flavor: default 2024-09-16T22:12:45.670 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T22:12:45.670 INFO:teuthology.task.install:extra packages: [] 2024-09-16T22:12:45.670 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-16T22:12:45.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T22:12:45.673 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-16T22:12:45.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T22:12:45.826 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-16T22:12:45.826 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-16T22:12:45.829 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-16T22:12:45.830 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-16T22:12:45.934 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T22:12:45.934 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:12:45.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T22:12:45.940 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T22:12:45.941 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:12:45.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T22:12:45.962 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T22:12:45.962 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T22:12:45.968 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T22:12:45.968 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T22:12:46.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T22:12:46.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T22:12:46.104 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T22:12:46.109 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T22:12:46.169 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-16T22:12:46.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-16T22:12:46.173 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-16T22:12:46.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-16T22:12:46.414 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-16T22:12:46.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-09-16T22:12:46.443 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-16T22:12:46.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-09-16T22:12:47.007 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-16T22:12:47.017 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-16T22:12:47.299 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-16T22:12:47.314 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 96 kB/s | 24 kB 00:00 2024-09-16T22:12:47.558 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 8.8 kB/s | 1.9 kB 00:00 2024-09-16T22:12:47.593 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T22:12:49.527 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-16T22:12:50.614 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.3 MB 00:03 2024-09-16T22:12:51.287 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-16T22:12:54.363 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 20 MB 00:02 2024-09-16T22:12:55.704 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.5 MB 00:00 2024-09-16T22:12:57.155 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-16T22:12:58.042 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T22:13:00.227 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-16T22:13:02.011 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-16T22:13:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-16T22:13:03.439 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T22:13:05.664 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T22:13:05.668 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T22:13:05.669 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-16T22:13:05.670 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:05.671 INFO:teuthology.orchestra.run.smithi092.stdout:Install 31 Packages 2024-09-16T22:13:05.672 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-09-16T22:13:05.672 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:05.672 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 62 M 2024-09-16T22:13:05.672 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:13:07.090 INFO:teuthology.orchestra.run.smithi092.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 5.9 MB/s | 5.5 MB 00:00 2024-09-16T22:13:07.132 INFO:teuthology.orchestra.run.smithi092.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 610 kB/s | 25 kB 00:00 2024-09-16T22:13:07.266 INFO:teuthology.orchestra.run.smithi092.stdout:(3/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 9.9 MB/s | 11 MB 00:01 2024-09-16T22:13:07.301 INFO:teuthology.orchestra.run.smithi092.stdout:(4/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 4.4 MB/s | 755 kB 00:00 2024-09-16T22:13:07.335 INFO:teuthology.orchestra.run.smithi092.stdout:(5/33): libradosstriper1-19.3.0-4883.g6e8474a0. 7.2 MB/s | 508 kB 00:00 2024-09-16T22:13:07.394 INFO:teuthology.orchestra.run.smithi092.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 775 kB/s | 45 kB 00:00 2024-09-16T22:13:07.461 INFO:teuthology.orchestra.run.smithi092.stdout:(7/33): python3-ceph-common-19.3.0-4883.g6e8474 2.1 MB/s | 143 kB 00:00 2024-09-16T22:13:07.703 INFO:teuthology.orchestra.run.smithi092.stdout:(8/33): ceph-common-19.3.0-4883.g6e8474a0.el9.x 14 MB/s | 22 MB 00:01 2024-09-16T22:13:07.727 INFO:teuthology.orchestra.run.smithi092.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 607 kB/s | 162 kB 00:00 2024-09-16T22:13:07.753 INFO:teuthology.orchestra.run.smithi092.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 6.3 MB/s | 320 kB 00:00 2024-09-16T22:13:07.778 INFO:teuthology.orchestra.run.smithi092.stdout:(11/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 6.0 MB/s | 307 kB 00:00 2024-09-16T22:13:07.862 INFO:teuthology.orchestra.run.smithi092.stdout:(12/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_ 9.9 MB/s | 5.5 MB 00:00 2024-09-16T22:13:07.887 INFO:teuthology.orchestra.run.smithi092.stdout:(13/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 743 kB/s | 99 kB 00:00 2024-09-16T22:13:08.004 INFO:teuthology.orchestra.run.smithi092.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-16T22:13:08.029 INFO:teuthology.orchestra.run.smithi092.stdout:(15/33): boost-program-options-1.75.0-8.el9.x86 426 kB/s | 107 kB 00:00 2024-09-16T22:13:08.054 INFO:teuthology.orchestra.run.smithi092.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-16T22:13:08.080 INFO:teuthology.orchestra.run.smithi092.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-16T22:13:08.138 INFO:teuthology.orchestra.run.smithi092.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-16T22:13:08.164 INFO:teuthology.orchestra.run.smithi092.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 506 kB/s | 42 kB 00:00 2024-09-16T22:13:08.189 INFO:teuthology.orchestra.run.smithi092.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-16T22:13:08.222 INFO:teuthology.orchestra.run.smithi092.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-16T22:13:08.247 INFO:teuthology.orchestra.run.smithi092.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 429 kB/s | 36 kB 00:00 2024-09-16T22:13:08.273 INFO:teuthology.orchestra.run.smithi092.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-16T22:13:08.373 INFO:teuthology.orchestra.run.smithi092.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-16T22:13:08.398 INFO:teuthology.orchestra.run.smithi092.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 235 kB/s | 26 kB 00:00 2024-09-16T22:13:08.423 INFO:teuthology.orchestra.run.smithi092.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-16T22:13:08.449 INFO:teuthology.orchestra.run.smithi092.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T22:13:08.482 INFO:teuthology.orchestra.run.smithi092.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T22:13:08.516 INFO:teuthology.orchestra.run.smithi092.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T22:13:08.591 INFO:teuthology.orchestra.run.smithi092.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T22:13:08.629 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-16T22:13:08.683 INFO:teuthology.orchestra.run.smithi092.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-16T22:13:08.792 INFO:teuthology.orchestra.run.smithi092.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-16T22:13:08.867 INFO:teuthology.orchestra.run.smithi092.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 12 MB/s | 3.2 MB 00:00 2024-09-16T22:13:08.871 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:08.872 INFO:teuthology.orchestra.run.smithi092.stdout:Total 19 MB/s | 62 MB 00:03 2024-09-16T22:13:08.982 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:13:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:13:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:13:09.695 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:13:09.695 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:13:10.967 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:13:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-16T22:13:10.981 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-16T22:13:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:Install 31 Packages 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-09-16T22:13:10.983 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:10.984 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 62 M 2024-09-16T22:13:10.984 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:13:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:13:11.293 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T22:13:11.322 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T22:13:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T22:13:11.370 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T22:13:11.391 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T22:13:11.589 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T22:13:11.641 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T22:13:11.998 INFO:teuthology.orchestra.run.smithi031.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 10 MB/s | 5.5 MB 00:00 2024-09-16T22:13:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T22:13:12.056 INFO:teuthology.orchestra.run.smithi031.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 432 kB/s | 25 kB 00:00 2024-09-16T22:13:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:12.173 INFO:teuthology.orchestra.run.smithi031.stdout:(3/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 6.3 MB/s | 755 kB 00:00 2024-09-16T22:13:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:12.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-16T22:13:12.290 INFO:teuthology.orchestra.run.smithi031.stdout:(4/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 13 MB/s | 11 MB 00:00 2024-09-16T22:13:12.291 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T22:13:12.324 INFO:teuthology.orchestra.run.smithi031.stdout:(5/33): libradosstriper1-19.3.0-4883.g6e8474a0. 3.3 MB/s | 508 kB 00:00 2024-09-16T22:13:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T22:13:12.382 INFO:teuthology.orchestra.run.smithi031.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 776 kB/s | 45 kB 00:00 2024-09-16T22:13:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T22:13:12.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-16T22:13:12.483 INFO:teuthology.orchestra.run.smithi031.stdout:(7/33): python3-ceph-common-19.3.0-4883.g6e8474 1.4 MB/s | 143 kB 00:00 2024-09-16T22:13:12.544 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T22:13:12.569 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T22:13:12.583 INFO:teuthology.orchestra.run.smithi031.stdout:(8/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-16T22:13:12.608 INFO:teuthology.orchestra.run.smithi031.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 1.3 MB/s | 162 kB 00:00 2024-09-16T22:13:12.633 INFO:teuthology.orchestra.run.smithi031.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 6.2 MB/s | 320 kB 00:00 2024-09-16T22:13:12.636 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T22:13:12.658 INFO:teuthology.orchestra.run.smithi031.stdout:(11/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 6.0 MB/s | 307 kB 00:00 2024-09-16T22:13:12.671 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T22:13:12.684 INFO:teuthology.orchestra.run.smithi031.stdout:(12/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T22:13:12.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T22:13:12.888 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T22:13:12.901 INFO:teuthology.orchestra.run.smithi031.stdout:(13/33): ceph-common-19.3.0-4883.g6e8474a0.el9. 15 MB/s | 22 MB 00:01 2024-09-16T22:13:12.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T22:13:12.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T22:13:12.996 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T22:13:13.029 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T22:13:13.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T22:13:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 171 kB/s | 107 kB 00:00 2024-09-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi031.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 232 kB/s | 171 kB 00:00 2024-09-16T22:13:13.425 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T22:13:13.443 INFO:teuthology.orchestra.run.smithi031.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-09-16T22:13:13.468 INFO:teuthology.orchestra.run.smithi031.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 282 kB/s | 160 kB 00:00 2024-09-16T22:13:13.524 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T22:13:13.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-16T22:13:13.585 INFO:teuthology.orchestra.run.smithi031.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-16T22:13:13.608 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T22:13:13.786 INFO:teuthology.orchestra.run.smithi031.stdout:(19/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-16T22:13:13.810 INFO:teuthology.orchestra.run.smithi031.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-09-16T22:13:13.836 INFO:teuthology.orchestra.run.smithi031.stdout:(21/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 745 kB/s | 292 kB 00:00 2024-09-16T22:13:13.886 INFO:teuthology.orchestra.run.smithi031.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-16T22:13:13.911 INFO:teuthology.orchestra.run.smithi031.stdout:(23/33): qatlib-service-24.02.0-1.el9.x86_64.rp 285 kB/s | 36 kB 00:00 2024-09-16T22:13:13.937 INFO:teuthology.orchestra.run.smithi031.stdout:(24/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-09-16T22:13:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-09-16T22:13:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-09-16T22:13:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-16T22:13:14.080 INFO:teuthology.orchestra.run.smithi031.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T22:13:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T22:13:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-16T22:13:14.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T22:13:14.205 INFO:teuthology.orchestra.run.smithi031.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-16T22:13:14.216 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T22:13:14.247 INFO:teuthology.orchestra.run.smithi031.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-16T22:13:14.294 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-16T22:13:14.321 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-16T22:13:14.397 INFO:teuthology.orchestra.run.smithi031.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-16T22:13:14.464 INFO:teuthology.orchestra.run.smithi031.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-16T22:13:14.469 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:14.470 INFO:teuthology.orchestra.run.smithi031.stdout:Total 18 MB/s | 62 MB 00:03 2024-09-16T22:13:14.588 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:13:14.629 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:13:14.629 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:13:15.335 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:13:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:16.871 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:13:17.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T22:13:17.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T22:13:17.283 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T22:13:17.283 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T22:13:17.283 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T22:13:17.283 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T22:13:17.283 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:17.329 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T22:13:17.332 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T22:13:17.414 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T22:13:17.414 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T22:13:17.414 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T22:13:17.414 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:17.447 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T22:13:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:17.905 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-16T22:13:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T22:13:18.110 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-16T22:13:18.139 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T22:13:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-16T22:13:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T22:13:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T22:13:18.331 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-16T22:13:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T22:13:18.477 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T22:13:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T22:13:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T22:13:18.673 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T22:13:18.702 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T22:13:18.735 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T22:13:18.781 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T22:13:19.128 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T22:13:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T22:13:19.241 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-16T22:13:19.270 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-16T22:13:19.306 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T22:13:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T22:13:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T22:13:19.922 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-16T22:13:20.003 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-16T22:13:20.036 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-16T22:13:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-16T22:13:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T22:13:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-16T22:13:23.204 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T22:13:23.204 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T22:13:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-16T22:13:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:28.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T22:13:28.362 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T22:13:28.362 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:28.362 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T22:13:28.363 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T22:13:28.363 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T22:13:28.363 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:28.364 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T22:13:28.431 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T22:13:28.431 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:29.655 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:29.655 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-16T22:13:29.656 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T22:13:29.657 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T22:13:29.658 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T22:13:29.658 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T22:13:29.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-16T22:13:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T22:13:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T22:13:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:30.630 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:13:30.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-09-16T22:13:31.567 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:13:31.698 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T22:13:31.699 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 51 M 2024-09-16T22:13:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 217 M 2024-09-16T22:13:31.701 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:13:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T22:13:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 689 kB/s | 161 kB 00:00 2024-09-16T22:13:31.985 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T22:13:33.285 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T22:13:33.287 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:33.287 INFO:teuthology.orchestra.run.smithi092.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-16T22:13:33.317 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:13:33.336 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:13:33.336 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-16T22:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-16T22:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-16T22:13:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-16T22:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-16T22:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-16T22:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-16T22:13:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:33.714 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:13:33.714 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:13:34.254 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:13:34.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T22:13:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T22:13:34.412 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T22:13:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T22:13:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T22:13:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T22:13:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-16T22:13:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T22:13:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T22:13:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-16T22:13:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T22:13:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-16T22:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T22:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-16T22:13:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T22:13:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:37.080 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:13:37.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-09-16T22:13:38.021 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:30 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:13:38.157 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:13:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:38.159 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 51 M 2024-09-16T22:13:38.160 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 217 M 2024-09-16T22:13:38.160 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:13:38.634 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 740 kB/s | 161 kB 00:00 2024-09-16T22:13:38.660 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-16T22:13:38.768 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 477 kB/s | 64 kB 00:00 2024-09-16T22:13:39.138 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T22:13:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T22:13:40.002 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T22:13:40.004 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:40.004 INFO:teuthology.orchestra.run.smithi031.stdout:Total 28 MB/s | 51 MB 00:01 2024-09-16T22:13:40.033 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:13:40.055 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:13:40.055 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:13:40.455 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:13:40.456 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:13:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:13:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T22:13:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T22:13:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T22:13:41.732 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T22:13:41.732 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-16T22:13:41.732 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-16T22:13:41.732 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T22:13:42.021 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:42.022 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:13:42.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-09-16T22:13:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:13:43.092 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:13:43.095 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T22:13:43.096 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T22:13:43.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T22:13:43.098 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:43.099 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout:Install 37 Packages 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 31 M 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 109 M 2024-09-16T22:13:43.100 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:13:43.426 INFO:teuthology.orchestra.run.smithi092.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 56 kB/s | 6.6 kB 00:00 2024-09-16T22:13:43.635 INFO:teuthology.orchestra.run.smithi092.stdout:(2/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 7.3 MB/s | 2.4 MB 00:00 2024-09-16T22:13:43.684 INFO:teuthology.orchestra.run.smithi092.stdout:(3/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-09-16T22:13:43.785 INFO:teuthology.orchestra.run.smithi092.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 2.8 MB/s | 291 kB 00:00 2024-09-16T22:13:43.852 INFO:teuthology.orchestra.run.smithi092.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T22:13:44.019 INFO:teuthology.orchestra.run.smithi092.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-16T22:13:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:(7/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-16T22:13:44.227 INFO:teuthology.orchestra.run.smithi092.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 511 kB/s | 106 kB 00:00 2024-09-16T22:13:44.303 INFO:teuthology.orchestra.run.smithi092.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-16T22:13:44.370 INFO:teuthology.orchestra.run.smithi092.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-09-16T22:13:44.395 INFO:teuthology.orchestra.run.smithi092.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 703 kB/s | 135 kB 00:00 2024-09-16T22:13:44.420 INFO:teuthology.orchestra.run.smithi092.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-16T22:13:44.520 INFO:teuthology.orchestra.run.smithi092.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 348 kB/s | 35 kB 00:00 2024-09-16T22:13:44.554 INFO:teuthology.orchestra.run.smithi092.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-16T22:13:44.579 INFO:teuthology.orchestra.run.smithi092.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 901 kB/s | 188 kB 00:00 2024-09-16T22:13:44.604 INFO:teuthology.orchestra.run.smithi092.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T22:13:44.630 INFO:teuthology.orchestra.run.smithi092.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 707 kB/s | 77 kB 00:00 2024-09-16T22:13:44.655 INFO:teuthology.orchestra.run.smithi092.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-16T22:13:44.681 INFO:teuthology.orchestra.run.smithi092.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 294 kB/s | 22 kB 00:00 2024-09-16T22:13:44.739 INFO:teuthology.orchestra.run.smithi092.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-09-16T22:13:44.764 INFO:teuthology.orchestra.run.smithi092.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-16T22:13:44.798 INFO:teuthology.orchestra.run.smithi092.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 323 kB/s | 11 kB 00:00 2024-09-16T22:13:44.823 INFO:teuthology.orchestra.run.smithi092.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T22:13:44.849 INFO:teuthology.orchestra.run.smithi092.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-09-16T22:13:44.874 INFO:teuthology.orchestra.run.smithi092.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-16T22:13:44.899 INFO:teuthology.orchestra.run.smithi092.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-16T22:13:44.924 INFO:teuthology.orchestra.run.smithi092.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-16T22:13:44.941 INFO:teuthology.orchestra.run.smithi092.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-16T22:13:44.966 INFO:teuthology.orchestra.run.smithi092.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-16T22:13:44.983 INFO:teuthology.orchestra.run.smithi092.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-16T22:13:45.000 INFO:teuthology.orchestra.run.smithi092.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 489 kB/s | 16 kB 00:00 2024-09-16T22:13:45.017 INFO:teuthology.orchestra.run.smithi092.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-16T22:13:45.042 INFO:teuthology.orchestra.run.smithi092.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-16T22:13:45.068 INFO:teuthology.orchestra.run.smithi092.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-16T22:13:45.101 INFO:teuthology.orchestra.run.smithi092.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T22:13:45.138 INFO:teuthology.orchestra.run.smithi092.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 286 kB/s | 20 kB 00:00 2024-09-16T22:13:45.163 INFO:teuthology.orchestra.run.smithi092.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.5 MB/s | 427 kB 00:00 2024-09-16T22:13:45.167 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:45.168 INFO:teuthology.orchestra.run.smithi092.stdout:Total 15 MB/s | 31 MB 00:02 2024-09-16T22:13:45.318 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:13:45.358 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:13:45.359 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:13:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:13:45.830 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:13:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-16T22:13:46.289 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T22:13:47.200 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:13:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T22:13:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T22:13:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T22:13:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T22:13:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T22:13:47.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T22:13:47.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T22:13:47.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T22:13:47.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T22:13:47.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T22:13:47.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T22:13:47.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T22:13:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T22:13:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T22:13:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T22:13:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T22:13:48.002 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T22:13:48.034 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T22:13:48.107 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T22:13:48.245 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T22:13:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T22:13:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T22:13:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T22:13:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T22:13:48.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T22:13:48.710 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T22:13:48.762 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T22:13:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-16T22:13:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-16T22:13:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-16T22:13:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T22:13:48.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T22:13:48.929 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T22:13:48.968 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T22:13:49.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T22:13:49.259 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-16T22:13:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T22:13:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:49.261 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:13:49.287 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T22:13:49.314 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T22:13:49.314 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:49.314 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T22:13:49.315 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T22:13:49.315 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T22:13:49.315 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:49.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-09-16T22:13:50.160 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:42 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:13:50.304 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-16T22:13:50.307 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections 2024-09-16T22:13:50.308 INFO:teuthology.orchestra.run.smithi031.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T22:13:50.309 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout:Install 37 Packages 2024-09-16T22:13:50.310 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:50.311 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 31 M 2024-09-16T22:13:50.311 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 109 M 2024-09-16T22:13:50.311 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:13:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T22:13:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T22:13:50.790 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:50.791 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T22:13:50.791 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T22:13:50.791 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T22:13:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T22:13:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:51.355 INFO:teuthology.orchestra.run.smithi031.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 55 kB/s | 6.6 kB 00:00 2024-09-16T22:13:51.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T22:13:51.521 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T22:13:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T22:13:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T22:13:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T22:13:51.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T22:13:51.580 INFO:teuthology.orchestra.run.smithi031.stdout:(2/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 6.9 MB/s | 2.4 MB 00:00 2024-09-16T22:13:51.622 INFO:teuthology.orchestra.run.smithi031.stdout:(3/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 3.9 MB/s | 1.5 MB 00:00 2024-09-16T22:13:51.714 INFO:teuthology.orchestra.run.smithi031.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 3.1 MB/s | 291 kB 00:00 2024-09-16T22:13:51.797 INFO:teuthology.orchestra.run.smithi031.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T22:13:51.973 INFO:teuthology.orchestra.run.smithi031.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 980 kB/s | 253 kB 00:00 2024-09-16T22:13:52.140 INFO:teuthology.orchestra.run.smithi031.stdout:(7/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-16T22:13:52.165 INFO:teuthology.orchestra.run.smithi031.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-09-16T22:13:52.215 INFO:teuthology.orchestra.run.smithi031.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T22:13:52.265 INFO:teuthology.orchestra.run.smithi031.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-16T22:13:52.291 INFO:teuthology.orchestra.run.smithi031.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-16T22:13:52.341 INFO:teuthology.orchestra.run.smithi031.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 673 kB/s | 135 kB 00:00 2024-09-16T22:13:52.933 INFO:teuthology.orchestra.run.smithi031.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 59 kB/s | 35 kB 00:00 2024-09-16T22:13:53.000 INFO:teuthology.orchestra.run.smithi031.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-16T22:13:53.025 INFO:teuthology.orchestra.run.smithi031.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-09-16T22:13:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T22:13:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-16T22:13:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-16T22:13:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-16T22:13:53.414 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T22:13:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T22:13:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T22:13:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T22:13:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T22:13:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T22:13:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T22:13:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T22:13:53.600 INFO:teuthology.orchestra.run.smithi031.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-16T22:13:53.625 INFO:teuthology.orchestra.run.smithi031.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 69 kB/s | 42 kB 00:00 2024-09-16T22:13:53.651 INFO:teuthology.orchestra.run.smithi031.stdout:(18/37): python3-packaging-20.9-5.el9.noarch.rp 108 kB/s | 77 kB 00:00 2024-09-16T22:13:53.701 INFO:teuthology.orchestra.run.smithi031.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-16T22:13:53.734 INFO:teuthology.orchestra.run.smithi031.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 520 kB/s | 43 kB 00:00 2024-09-16T22:13:53.760 INFO:teuthology.orchestra.run.smithi031.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.0 MB/s | 172 kB 00:00 2024-09-16T22:13:53.785 INFO:teuthology.orchestra.run.smithi031.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-16T22:13:53.836 INFO:teuthology.orchestra.run.smithi031.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-16T22:13:53.861 INFO:teuthology.orchestra.run.smithi031.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 930 kB/s | 23 kB 00:00 2024-09-16T22:13:53.895 INFO:teuthology.orchestra.run.smithi031.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-16T22:13:53.920 INFO:teuthology.orchestra.run.smithi031.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 333 kB/s | 19 kB 00:00 2024-09-16T22:13:53.945 INFO:teuthology.orchestra.run.smithi031.stdout:(27/37): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-09-16T22:13:53.970 INFO:teuthology.orchestra.run.smithi031.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-16T22:13:54.004 INFO:teuthology.orchestra.run.smithi031.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 552 kB/s | 46 kB 00:00 2024-09-16T22:13:54.029 INFO:teuthology.orchestra.run.smithi031.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 721 kB/s | 16 kB 00:00 2024-09-16T22:13:54.062 INFO:teuthology.orchestra.run.smithi031.stdout:(31/37): python3-more-itertools-8.12.0-2.el9.no 673 kB/s | 79 kB 00:00 2024-09-16T22:13:54.088 INFO:teuthology.orchestra.run.smithi031.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-16T22:13:54.113 INFO:teuthology.orchestra.run.smithi031.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-16T22:13:54.147 INFO:teuthology.orchestra.run.smithi031.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-16T22:13:54.172 INFO:teuthology.orchestra.run.smithi031.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 807 kB/s | 20 kB 00:00 2024-09-16T22:13:54.197 INFO:teuthology.orchestra.run.smithi031.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T22:13:54.222 INFO:teuthology.orchestra.run.smithi031.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T22:13:54.227 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:54.227 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.8 MB/s | 31 MB 00:03 2024-09-16T22:13:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:13:54.434 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:13:54.434 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:13:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T22:13:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:54.439 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:13:54.439 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T22:13:54.440 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T22:13:54.441 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T22:13:54.442 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T22:13:54.443 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:54.443 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:13:54.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-09-16T22:13:54.921 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:13:54.921 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:13:55.370 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:13:55.458 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:13:55.499 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:55.500 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:13:55.501 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:13:55.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-09-16T22:13:56.141 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:13:56.271 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:56.272 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 784 k 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 791 k 2024-09-16T22:13:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:13:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:13:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T22:13:56.530 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T22:13:56.553 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-16T22:13:56.554 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:56.554 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-16T22:13:56.554 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:13:56.559 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:13:56.560 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:13:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T22:13:56.566 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:13:56.566 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:13:56.636 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T22:13:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:13:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T22:13:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T22:13:56.807 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T22:13:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T22:13:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T22:13:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T22:13:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:13:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T22:13:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T22:13:57.196 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T22:13:57.198 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:13:57.246 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T22:13:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T22:13:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T22:13:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T22:13:57.412 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T22:13:57.489 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T22:13:57.583 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:13:57.641 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T22:13:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T22:13:57.791 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T22:13:57.837 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:57.839 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:13:57.921 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T22:13:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T22:13:58.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T22:13:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T22:13:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T22:13:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T22:13:58.403 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T22:13:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T22:13:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T22:13:58.642 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:13:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-16T22:13:58.774 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:13:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:13:58.776 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:13:58.776 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 145 k 2024-09-16T22:13:58.776 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 432 k 2024-09-16T22:13:58.777 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:13:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T22:13:58.827 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:13:58.969 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 752 kB/s | 145 kB 00:00 2024-09-16T22:13:58.970 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:13:58.970 INFO:teuthology.orchestra.run.smithi092.stdout:Total 744 kB/s | 145 kB 00:00 2024-09-16T22:13:58.970 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:13:58.978 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:13:58.978 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:13:59.035 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:13:59.035 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:13:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:13:59.270 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:13:59.294 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:13:59.294 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:13:59.295 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T22:13:59.295 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:14:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:00.213 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:00.214 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:00.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T22:14:00.404 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:00.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-09-16T22:14:00.884 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T22:14:00.909 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-16T22:14:00.909 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:00.909 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T22:14:00.910 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T22:14:00.910 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T22:14:00.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:01.078 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T22:14:01.169 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-16T22:14:01.176 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:01.176 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T22:14:01.177 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T22:14:01.177 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T22:14:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:01.211 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:01.212 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:01.212 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:01.242 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T22:14:01.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T22:14:01.856 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:01.986 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:01.987 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T22:14:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-09-16T22:14:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:01.990 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 19 M 2024-09-16T22:14:01.990 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 159 M 2024-09-16T22:14:01.990 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:02.438 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 216 kB/s | 34 kB 00:00 2024-09-16T22:14:02.463 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 90 kB/s | 16 kB 00:00 2024-09-16T22:14:02.605 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-16T22:14:02.656 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-09-16T22:14:02.714 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-16T22:14:02.739 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-16T22:14:02.815 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.5 MB/s | 863 kB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.1 MB/s | 144 kB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 524 kB/s | 31 kB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.8 MB/s | 188 kB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474 15 MB/s | 14 MB 00:00 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-16T22:14:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:03.266 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T22:14:03.267 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T22:14:03.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T22:14:03.269 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T22:14:03.270 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:03.271 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:03.550 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:03.551 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:04.163 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:04.165 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T22:14:04.165 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:04.165 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T22:14:04.166 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T22:14:04.167 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T22:14:04.168 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T22:14:04.169 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T22:14:04.169 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T22:14:04.169 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:04.169 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T22:14:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T22:14:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T22:14:04.354 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T22:14:04.457 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T22:14:04.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-09-16T22:14:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T22:14:04.673 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T22:14:04.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T22:14:04.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T22:14:04.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-16T22:14:05.166 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:05.259 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:05.303 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:05.304 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:05.305 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:05.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-09-16T22:14:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:06.129 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:06.130 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 784 k 2024-09-16T22:14:06.131 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 791 k 2024-09-16T22:14:06.132 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:06.408 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-16T22:14:06.409 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:06.409 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-16T22:14:06.409 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:06.415 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:06.415 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:06.421 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:06.421 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:06.501 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:06.973 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:07.081 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:07.721 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:07.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T22:14:07.966 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-16T22:14:08.023 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T22:14:08.534 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:08.674 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:08.675 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 145 k 2024-09-16T22:14:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 432 k 2024-09-16T22:14:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:08.880 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 707 kB/s | 145 kB 00:00 2024-09-16T22:14:08.881 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:08.881 INFO:teuthology.orchestra.run.smithi031.stdout:Total 703 kB/s | 145 kB 00:00 2024-09-16T22:14:08.881 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:08.889 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:08.889 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:08.951 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:08.951 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:14:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:14:09.200 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:09.200 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T22:14:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T22:14:09.977 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T22:14:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T22:14:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T22:14:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T22:14:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T22:14:09.978 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:10.296 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T22:14:10.297 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T22:14:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T22:14:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T22:14:10.298 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:10.298 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:10.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-09-16T22:14:10.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T22:14:11.006 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:03 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:11.098 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:11.125 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:08 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:11.143 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:11.144 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:11.144 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:11.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T22:14:11.257 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:11.258 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-16T22:14:11.258 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:11.258 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-16T22:14:11.258 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:11.258 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T22:14:11.259 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-09-16T22:14:11.260 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:11.261 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-09-16T22:14:11.261 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 212 M 2024-09-16T22:14:11.261 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:11.671 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-16T22:14:11.704 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-16T22:14:11.788 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-16T22:14:11.817 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:03 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:11.841 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-16T22:14:11.889 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-16T22:14:11.954 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-16T22:14:11.955 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-16T22:14:11.957 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 19 M 2024-09-16T22:14:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 159 M 2024-09-16T22:14:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-48 15 MB/s | 7.4 MB 00:00 2024-09-16T22:14:12.189 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-16T22:14:12.589 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.4 MB/s | 3.0 MB 00:00 2024-09-16T22:14:12.673 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-16T22:14:12.708 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 203 kB/s | 34 kB 00:00 2024-09-16T22:14:12.733 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 85 kB/s | 16 kB 00:00 2024-09-16T22:14:12.850 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-16T22:14:12.901 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-16T22:14:12.976 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T22:14:12.993 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-16T22:14:13.048 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.3 MB/s | 4.9 MB 00:01 2024-09-16T22:14:13.076 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 25 MB/s | 2.0 MB 00:00 2024-09-16T22:14:13.110 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-16T22:14:13.135 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-16T22:14:13.160 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-16T22:14:13.311 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474 18 MB/s | 14 MB 00:00 2024-09-16T22:14:13.348 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-16T22:14:13.661 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.0 MB/s | 863 kB 00:00 2024-09-16T22:14:13.665 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:13.665 INFO:teuthology.orchestra.run.smithi031.stdout:Total 11 MB/s | 19 MB 00:01 2024-09-16T22:14:13.720 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:13.742 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:13.742 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T22:14:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T22:14:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T22:14:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T22:14:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-09-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi092.stdout:Total 10 MB/s | 38 MB 00:03 2024-09-16T22:14:15.098 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T22:14:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T22:14:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T22:14:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T22:14:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-16T22:14:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:15.400 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:15.400 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:15.850 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:15.850 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:16.617 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T22:14:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T22:14:16.839 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T22:14:16.872 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T22:14:17.058 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T22:14:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T22:14:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T22:14:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T22:14:17.968 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T22:14:18.698 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-16T22:14:18.751 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T22:14:19.257 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T22:14:20.307 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T22:14:20.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T22:14:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T22:14:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T22:14:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T22:14:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:21.061 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:21.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T22:14:21.924 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:13 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:22.065 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:14:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T22:14:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-09-16T22:14:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 212 M 2024-09-16T22:14:22.069 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-16T22:14:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-16T22:14:22.709 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-09-16T22:14:22.751 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-16T22:14:22.826 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-09-16T22:14:22.868 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 330 kB/s | 38 kB 00:00 2024-09-16T22:14:22.986 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T22:14:22.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-16T22:14:22.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T22:14:22.987 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T22:14:22.993 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): ceph-mgr-diskprediction-local-19.3.0-48 14 MB/s | 7.4 MB 00:00 2024-09-16T22:14:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-16T22:14:23.102 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2024-09-16T22:14:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-16T22:14:23.395 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T22:14:23.396 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T22:14:23.397 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T22:14:23.397 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T22:14:23.397 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T22:14:23.397 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:23.397 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-09-16T22:14:23.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-09-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.7 MB/s | 5.1 MB 00:00 2024-09-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi031.stdout:Total 21 MB/s | 38 MB 00:01 2024-09-16T22:14:24.270 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:24.298 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:24.298 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:24.416 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:21 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:24.549 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:24.550 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:24.550 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:24.550 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T22:14:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-09-16T22:14:24.552 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:24.754 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:24.755 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:24.878 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 294 kB/s | 49 kB 00:00 2024-09-16T22:14:25.253 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 35 kB/s | 19 kB 00:00 2024-09-16T22:14:25.278 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 45 kB/s | 26 kB 00:00 2024-09-16T22:14:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T22:14:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T22:14:25.703 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 269 kB/s | 222 kB 00:00 2024-09-16T22:14:25.790 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T22:14:25.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T22:14:25.979 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-09-16T22:14:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T22:14:26.037 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 555 kB/s | 32 kB 00:00 2024-09-16T22:14:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T22:14:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 356 kB/s | 279 kB 00:00 2024-09-16T22:14:26.088 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 285 kB/s | 14 kB 00:00 2024-09-16T22:14:26.130 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T22:14:26.140 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.8 MB/s | 220 kB 00:00 2024-09-16T22:14:26.172 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.0 MB/s | 59 kB 00:00 2024-09-16T22:14:26.197 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-16T22:14:26.239 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T22:14:26.397 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 77 kB/s | 54 kB 00:00 2024-09-16T22:14:26.401 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:26.401 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-16T22:14:26.473 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:26.491 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:26.491 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:26.587 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:26.588 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T22:14:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T22:14:27.089 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T22:14:27.332 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T22:14:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T22:14:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T22:14:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T22:14:27.566 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T22:14:27.669 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T22:14:27.726 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T22:14:28.275 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T22:14:28.448 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T22:14:28.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T22:14:28.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T22:14:28.588 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T22:14:28.609 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T22:14:29.339 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T22:14:29.340 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T22:14:29.340 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T22:14:29.340 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T22:14:29.340 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T22:14:29.340 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T22:14:29.371 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T22:14:29.400 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T22:14:29.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:29.666 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:29.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T22:14:30.527 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:30.658 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:30.658 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:30.659 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:30.660 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:30.660 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-09-16T22:14:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:30.660 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-09-16T22:14:30.660 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-09-16T22:14:30.661 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:31.382 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-16T22:14:31.407 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 729 kB/s | 154 kB 00:00 2024-09-16T22:14:31.457 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T22:14:31.490 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.9 MB/s | 548 kB 00:00 2024-09-16T22:14:31.516 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T22:14:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T22:14:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T22:14:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T22:14:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T22:14:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T22:14:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T22:14:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T22:14:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:32.409 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:32.410 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:32.683 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-09-16T22:14:32.685 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:32.685 INFO:teuthology.orchestra.run.smithi092.stdout:Total 3.5 MB/s | 7.0 MB 00:02 2024-09-16T22:14:32.756 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:32.767 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:32.767 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:32.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-09-16T22:14:32.868 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:32.868 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T22:14:33.392 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:33.409 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T22:14:33.531 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T22:14:33.533 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-09-16T22:14:33.534 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:33.535 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-09-16T22:14:33.535 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-09-16T22:14:33.535 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:33.716 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 267 kB/s | 26 kB 00:00 2024-09-16T22:14:33.827 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 158 kB/s | 19 kB 00:00 2024-09-16T22:14:33.827 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 274 kB/s | 49 kB 00:00 2024-09-16T22:14:33.834 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-16T22:14:33.875 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T22:14:33.901 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 800 kB/s | 54 kB 00:00 2024-09-16T22:14:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T22:14:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 428 kB/s | 32 kB 00:00 2024-09-16T22:14:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 188 kB/s | 14 kB 00:00 2024-09-16T22:14:34.001 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-16T22:14:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T22:14:34.035 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T22:14:34.060 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-16T22:14:34.085 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-09-16T22:14:34.089 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T22:14:34.113 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T22:14:34.119 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-16T22:14:34.122 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:34.122 INFO:teuthology.orchestra.run.smithi031.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-16T22:14:34.189 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:34.205 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:34.205 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:34.303 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:34.303 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T22:14:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-16T22:14:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T22:14:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T22:14:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T22:14:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T22:14:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:35.009 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T22:14:35.064 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T22:14:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T22:14:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:35.096 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:35.097 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:35.129 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T22:14:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T22:14:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T22:14:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T22:14:35.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-09-16T22:14:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T22:14:35.474 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T22:14:35.933 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:36.067 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-16T22:14:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 979 k 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.9 M 2024-09-16T22:14:36.069 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:36.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T22:14:36.303 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T22:14:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T22:14:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T22:14:36.392 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 3.2 MB/s | 899 kB 00:00 2024-09-16T22:14:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T22:14:36.534 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 191 kB/s | 80 kB 00:00 2024-09-16T22:14:36.534 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:36.535 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.1 MB/s | 979 kB 00:00 2024-09-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:36.569 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:36.569 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:36.651 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:36.651 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:36.996 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T22:14:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T22:14:37.145 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T22:14:37.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:37.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T22:14:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T22:14:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-16T22:14:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T22:14:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:38.049 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T22:14:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:38.050 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:38.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-09-16T22:14:38.358 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:38.496 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T22:14:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-09-16T22:14:38.499 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-09-16T22:14:38.499 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:38.903 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:39.003 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 731 kB/s | 154 kB 00:00 2024-09-16T22:14:39.038 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-16T22:14:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 285 k 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.3 M 2024-09-16T22:14:39.040 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:39.087 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-16T22:14:39.120 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-09-16T22:14:39.146 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T22:14:39.254 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-16T22:14:39.255 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:39.255 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-16T22:14:39.255 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:39.263 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:39.264 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:39.296 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:39.296 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:39.429 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 390 kB/s | 248 kB 00:00 2024-09-16T22:14:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:39.597 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:39.598 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:39.598 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T22:14:39.598 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:40.063 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-16T22:14:40.065 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:40.065 INFO:teuthology.orchestra.run.smithi031.stdout:Total 4.5 MB/s | 7.0 MB 00:01 2024-09-16T22:14:40.141 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:40.154 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:40.154 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:40.261 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:40.262 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:40.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:40.553 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:40.553 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:40.554 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:40.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:40.554 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T22:14:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T22:14:40.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-09-16T22:14:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T22:14:41.442 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T22:14:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T22:14:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T22:14:41.578 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-16T22:14:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 127 k 2024-09-16T22:14:41.580 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 456 k 2024-09-16T22:14:41.581 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:41.792 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 599 kB/s | 127 kB 00:00 2024-09-16T22:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout:Total 595 kB/s | 127 kB 00:00 2024-09-16T22:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:41.798 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:41.798 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:41.862 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:41.862 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:42.014 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T22:14:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T22:14:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T22:14:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T22:14:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T22:14:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:42.573 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:42.791 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:42.792 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:42.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-09-16T22:14:43.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-09-16T22:14:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:43.585 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:43.586 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:43.586 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:43.586 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 979 k 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.9 M 2024-09-16T22:14:43.587 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:43.631 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:43.723 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:43.766 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:43.767 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:43.767 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:43.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-09-16T22:14:43.858 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-16T22:14:44.014 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 2.8 MB/s | 899 kB 00:00 2024-09-16T22:14:44.015 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:44.015 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.2 MB/s | 979 kB 00:00 2024-09-16T22:14:44.027 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:44.050 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:44.050 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:44.135 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:44.135 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:44.412 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T22:14:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-16T22:14:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 32 k 2024-09-16T22:14:44.544 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 142 k 2024-09-16T22:14:44.545 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:44.695 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 211 kB/s | 32 kB 00:00 2024-09-16T22:14:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:Total 209 kB/s | 32 kB 00:00 2024-09-16T22:14:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:44.699 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:44.699 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:44.729 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:44.729 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-16T22:14:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-16T22:14:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T22:14:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T22:14:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:45.487 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:45.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-09-16T22:14:45.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-09-16T22:14:46.268 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:46.359 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:46.365 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:46.402 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:46.403 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:46.403 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:46.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-09-16T22:14:46.505 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:46.505 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 285 k 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.3 M 2024-09-16T22:14:46.506 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:46.743 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-16T22:14:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-16T22:14:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:46.754 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:46.754 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:46.784 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:46.785 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:47.019 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:47.055 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:47.096 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:47.096 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T22:14:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:47.149 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:47.193 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:47.194 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:47.194 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:47.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-09-16T22:14:47.839 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:47.928 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:47.971 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:47.971 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:47.972 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:48.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-09-16T22:14:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-16T22:14:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:48.113 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-16T22:14:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:48.114 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:48.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-09-16T22:14:48.622 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:48.712 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:48.755 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:48.756 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:48.756 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:48.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-09-16T22:14:49.019 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:49.157 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:49.158 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:49.159 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-09-16T22:14:49.159 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-09-16T22:14:49.159 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:49.382 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 569 kB/s | 127 kB 00:00 2024-09-16T22:14:49.382 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:49.383 INFO:teuthology.orchestra.run.smithi031.stdout:Total 565 kB/s | 127 kB 00:00 2024-09-16T22:14:49.383 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:49.389 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:49.389 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:49.417 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:49.458 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:49.458 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:49.512 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:49.557 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:49.559 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:49.559 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:49.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-09-16T22:14:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:50.259 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:50.352 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:50.397 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:50.398 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-16T22:14:50.398 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:50.446 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:50.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-09-16T22:14:50.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-09-16T22:14:51.051 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:51.186 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:51.186 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:51.186 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 86 k 2024-09-16T22:14:51.187 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 231 k 2024-09-16T22:14:51.188 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:51.314 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:51.389 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 424 kB/s | 86 kB 00:00 2024-09-16T22:14:51.390 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:51.390 INFO:teuthology.orchestra.run.smithi092.stdout:Total 421 kB/s | 86 kB 00:00 2024-09-16T22:14:51.390 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:51.397 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:51.397 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:51.409 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:51.456 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:51.460 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:51.461 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:51.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-09-16T22:14:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:51.702 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:52.139 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:52.276 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 32 k 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 142 k 2024-09-16T22:14:52.277 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:52.380 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:52.380 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:52.381 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:52.381 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:52.381 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:52.381 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:52.436 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 200 kB/s | 32 kB 00:00 2024-09-16T22:14:52.436 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:52.437 INFO:teuthology.orchestra.run.smithi031.stdout:Total 199 kB/s | 32 kB 00:00 2024-09-16T22:14:52.437 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:52.440 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:52.441 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:52.452 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:52.452 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:52.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-09-16T22:14:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:53.210 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:53.279 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:53.345 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.2 M 2024-09-16T22:14:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 14 M 2024-09-16T22:14:53.347 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:53.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-09-16T22:14:53.692 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 9.2 MB/s | 3.2 MB 00:00 2024-09-16T22:14:53.693 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:53.693 INFO:teuthology.orchestra.run.smithi092.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-16T22:14:53.693 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:53.702 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:53.703 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:53.784 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:53.784 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:54.070 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:54.165 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:54.211 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:54.212 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:54.213 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:54.236 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:54.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-09-16T22:14:54.301 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T22:14:54.326 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:54.896 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:54.988 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:55.035 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:55.036 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:55.036 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:55.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-09-16T22:14:55.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:55.445 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:55.445 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:55.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-09-16T22:14:55.720 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:55.813 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:55.859 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:55.860 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:55.861 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:55.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-09-16T22:14:56.312 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:53 ago on Mon 16 Sep 2024 10:13:03 PM UTC. 2024-09-16T22:14:56.448 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 170 k 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 490 k 2024-09-16T22:14:56.449 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-16T22:14:56.547 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:56.653 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 836 kB/s | 170 kB 00:00 2024-09-16T22:14:56.654 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:56.654 INFO:teuthology.orchestra.run.smithi092.stdout:Total 830 kB/s | 170 kB 00:00 2024-09-16T22:14:56.654 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-16T22:14:56.662 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-16T22:14:56.662 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-16T22:14:56.685 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:56.686 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:56.686 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:56.725 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-16T22:14:56.726 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-16T22:14:56.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-09-16T22:14:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-16T22:14:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:57.366 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:49 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:57.419 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:57.458 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:57.504 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:57.505 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:57.505 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:57.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-09-16T22:14:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:14:57.686 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:57.686 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-16T22:14:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:14:57.687 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:14:57.687 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-16T22:14:57.924 DEBUG:teuthology.parallel:result is None 2024-09-16T22:14:58.188 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:58.281 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-16T22:14:58.327 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:58.328 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T22:14:58.328 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:14:58.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-09-16T22:14:59.001 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:51 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:14:59.143 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:14:59.144 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:14:59.144 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:14:59.144 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 86 k 2024-09-16T22:14:59.144 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 231 k 2024-09-16T22:14:59.144 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:14:59.339 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 439 kB/s | 86 kB 00:00 2024-09-16T22:14:59.340 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:14:59.340 INFO:teuthology.orchestra.run.smithi031.stdout:Total 435 kB/s | 86 kB 00:00 2024-09-16T22:14:59.340 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:14:59.348 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:14:59.348 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:14:59.415 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:14:59.416 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:14:59.570 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:14:59.681 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:00.058 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:00.322 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:00.322 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:00.322 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:15:00.323 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:15:00.323 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:00.323 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:15:00.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-09-16T22:15:01.213 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:53 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:15:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.2 M 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 14 M 2024-09-16T22:15:01.354 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:15:01.733 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 8.4 MB/s | 3.2 MB 00:00 2024-09-16T22:15:01.734 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:15:01.734 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.4 MB/s | 3.2 MB 00:00 2024-09-16T22:15:01.734 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:15:01.745 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:15:01.745 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:15:01.832 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:15:01.832 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:15:02.383 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:15:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T22:15:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:03.670 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:15:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:15:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:03.670 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:15:03.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-16T22:15:04.518 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:56 ago on Mon 16 Sep 2024 10:13:08 PM UTC. 2024-09-16T22:15:04.659 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:04.660 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 170 k 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 490 k 2024-09-16T22:15:04.661 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T22:15:04.870 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 816 kB/s | 170 kB 00:00 2024-09-16T22:15:04.870 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T22:15:04.870 INFO:teuthology.orchestra.run.smithi031.stdout:Total 811 kB/s | 170 kB 00:00 2024-09-16T22:15:04.871 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T22:15:04.879 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T22:15:04.879 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T22:15:04.947 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T22:15:04.948 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T22:15:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T22:15:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:05.886 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T22:15:06.142 DEBUG:teuthology.parallel:result is None 2024-09-16T22:15:06.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T22:15:06.289 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T22:15:06.311 INFO:teuthology.orchestra.run.smithi031.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-16T22:15:06.312 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-16T22:15:06.312 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-16T22:15:06.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-16T22:15:06.449 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T22:15:06.475 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-16T22:15:06.475 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-16T22:15:06.475 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-16T22:15:06.477 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T22:15:06.477 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:06.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T22:15:06.510 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:06.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T22:15:06.545 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T22:15:06.545 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:06.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T22:15:06.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T22:15:06.644 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:06.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T22:15:06.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T22:15:06.739 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T22:15:06.739 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:06.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T22:15:06.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T22:15:06.831 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:06.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T22:15:06.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T22:15:06.924 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T22:15:06.925 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:06.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T22:15:06.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T22:15:07.016 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:07.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T22:15:07.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T22:15:07.109 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T22:15:07.186 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T22:15:07.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-16T22:15:07.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-09-16T22:15:07.214 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T22:15:07.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T22:15:07.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T22:15:07.280 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T22:15:07.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T22:15:07.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T22:15:07.354 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T22:15:07.355 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T22:15:07.355 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596', 'branch': 'wip-dis-testing', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7907596', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2961, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'sleep_before_teardown': 0, 'subset': '66/100', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGORqTOgq1LQ4dFy+zGK+rqmgXrEzhhUuBDjzVadRA4VnDZbZEf1OYfiZXZDBbq0BpZEYhzJ5Qf6DOLnZYYgfs=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaY0tqkGzJZHIAvKvbsOPTEZPEwMoT2OpTXZ/fSCljvk5pHpnCHkLPcf95R4RXK41U+0g9CSqwE7p0PvAFbkyM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_19:45:14', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T22:15:07.355 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T22:15:07.386 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T22:15:07.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T22:15:07.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T22:15:07.467 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:07.467 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T22:15:07.522 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T22:15:07.522 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 22:15:02.748805492 +0000 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 22:12:25.727741837 +0000 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 22:12:25.727741837 +0000 2024-09-16T22:15:07.578 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 22:12:25.727741837 +0000 2024-09-16T22:15:07.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T22:15:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T22:15:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T22:15:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000257267 s, 2.0 MB/s 2024-09-16T22:15:07.643 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T22:15:07.700 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 22:15:02.748805492 +0000 2024-09-16T22:15:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 22:12:26.077736169 +0000 2024-09-16T22:15:07.757 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 22:12:26.077736169 +0000 2024-09-16T22:15:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 22:12:26.077736169 +0000 2024-09-16T22:15:07.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T22:15:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T22:15:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T22:15:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000228567 s, 2.2 MB/s 2024-09-16T22:15:07.821 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T22:15:07.878 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 22:15:02.749805475 +0000 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 22:12:26.419730630 +0000 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 22:12:26.419730630 +0000 2024-09-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 22:12:26.419730630 +0000 2024-09-16T22:15:07.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T22:15:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T22:15:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T22:15:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000210815 s, 2.4 MB/s 2024-09-16T22:15:08.000 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T22:15:08.056 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-16T22:15:08.111 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 22:15:02.749805475 +0000 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 22:12:26.765725027 +0000 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 22:12:26.765725027 +0000 2024-09-16T22:15:08.112 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 22:12:26.765725027 +0000 2024-09-16T22:15:08.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T22:15:08.176 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T22:15:08.176 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T22:15:08.176 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000210381 s, 2.4 MB/s 2024-09-16T22:15:08.178 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T22:15:08.235 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:08.235 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:08.235 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T22:15:08.254 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T22:15:08.254 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-16 22:14:54.579083336 +0000 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-16 22:12:19.854024513 +0000 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-16 22:12:19.855024485 +0000 2024-09-16T22:15:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-16 22:12:19.854024513 +0000 2024-09-16T22:15:08.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T22:15:08.377 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-16T22:15:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-16T22:15:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000214522 s, 2.4 MB/s 2024-09-16T22:15:08.379 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T22:15:08.437 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-16 22:14:54.580083308 +0000 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-16 22:12:20.183015252 +0000 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-16 22:12:20.183015252 +0000 2024-09-16T22:15:08.494 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-16 22:12:20.183015252 +0000 2024-09-16T22:15:08.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T22:15:08.559 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-16T22:15:08.560 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-16T22:15:08.560 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000219365 s, 2.3 MB/s 2024-09-16T22:15:08.561 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T22:15:08.618 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-16T22:15:08.674 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T22:15:08.674 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T22:15:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:08.674 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-16 22:14:54.580083308 +0000 2024-09-16T22:15:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-16 22:12:20.501006301 +0000 2024-09-16T22:15:08.675 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-16 22:12:20.501006301 +0000 2024-09-16T22:15:08.675 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-16 22:12:20.501006301 +0000 2024-09-16T22:15:08.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T22:15:08.741 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-16T22:15:08.742 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-16T22:15:08.742 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000217691 s, 2.4 MB/s 2024-09-16T22:15:08.742 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T22:15:08.798 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-16 22:14:54.580083308 +0000 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-16 22:12:20.811997547 +0000 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-16 22:12:20.811997547 +0000 2024-09-16T22:15:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-16 22:12:20.811997547 +0000 2024-09-16T22:15:08.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T22:15:08.917 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-16T22:15:08.918 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-16T22:15:08.918 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000190005 s, 2.7 MB/s 2024-09-16T22:15:08.919 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T22:15:08.975 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:08.975 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T22:15:08.975 INFO:tasks.ceph:Generating config... 2024-09-16T22:15:08.977 INFO:tasks.ceph:[client] rbd cache = False 2024-09-16T22:15:08.977 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T22:15:08.977 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T22:15:08.977 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T22:15:08.977 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T22:15:08.977 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T22:15:08.977 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T22:15:08.977 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T22:15:08.977 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-16T22:15:08.978 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T22:15:08.979 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T22:15:08.979 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-16T22:15:08.979 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T22:15:08.979 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T22:15:08.979 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T22:15:08.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T22:15:09.036 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T22:15:09.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T22:15:09.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T22:15:09.112 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.92')] 2024-09-16T22:15:09.113 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.92', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T22:15:09.113 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:09.114 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T22:15:09.169 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.92 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:09.257 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:09.257 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 57fe7da5-e356-48a4-ba33-5558b1026a32 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 57fe7da5-e356-48a4-ba33-5558b1026a32 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-09-16T22:15:09.260140+0000 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T22:15:09.260140+0000 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-16T22:15:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T22:15:09.259 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T22:15:09.275 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 57fe7da5-e356-48a4-ba33-5558b1026a32... 2024-09-16T22:15:09.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T22:15:09.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-16T22:15:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.92" 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 57fe7da5-e356-48a4-ba33-5558b1026a32 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.364 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.365 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2024-09-16T22:15:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: osd memory target = 1610612736 2024-09-16T22:15:09.367 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: osd shutdown pgref assert = True 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-16T22:15:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T22:15:09.369 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-16T22:15:09.370 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout: rbd cache = False 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-16T22:15:09.371 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.372 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-16T22:15:09.373 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-16T22:15:09.374 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-16T22:15:09.375 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.92" 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on pool no app = False 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2024-09-16T22:15:09.376 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 57fe7da5-e356-48a4-ba33-5558b1026a32 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-16T22:15:09.377 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-16T22:15:09.378 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-16T22:15:09.379 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd memory target = 1610612736 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: osd shutdown pgref assert = True 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-16T22:15:09.380 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-16T22:15:09.381 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-16T22:15:09.382 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T22:15:09.383 INFO:teuthology.orchestra.run.smithi092.stdout: rbd cache = False 2024-09-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-16T22:15:09.384 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-16T22:15:09.385 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T22:15:09.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T22:15:09.482 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T22:15:09.482 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:09.482 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T22:15:09.498 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:09.498 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T22:15:09.554 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:15:09.554 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:09.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T22:15:09.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T22:15:09.630 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:09.630 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:09.686 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:15:09.686 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:09.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T22:15:09.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T22:15:09.721 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:09.721 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:09.777 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T22:15:09.778 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T22:15:09.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T22:15:09.830 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T22:15:09.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T22:15:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T22:15:09.918 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T22:15:09.918 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T22:15:09.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T22:15:09.961 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T22:15:09.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T22:15:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T22:15:10.028 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T22:15:10.028 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T22:15:10.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T22:15:10.054 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:10.054 INFO:tasks.ceph:role: osd.0 2024-09-16T22:15:10.054 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:15:10.054 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:10.632 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-16T22:15:10.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-16T22:15:10.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T22:15:10.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T22:15:10.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T22:15:10.861 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:10.861 INFO:tasks.ceph:role: osd.1 2024-09-16T22:15:10.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:15:10.862 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T22:15:10.935 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:10.936 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:11.391 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-16T22:15:11.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-16T22:15:11.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T22:15:11.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T22:15:11.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T22:15:11.580 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:11.580 INFO:tasks.ceph:role: osd.2 2024-09-16T22:15:11.580 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:15:11.580 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T22:15:11.653 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:11.654 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:12.122 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-16T22:15:12.124 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-16T22:15:12.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T22:15:12.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T22:15:12.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T22:15:12.360 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:12.360 INFO:tasks.ceph:role: osd.3 2024-09-16T22:15:12.360 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:15:12.361 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:12.434 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:12.435 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:12.435 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:12.435 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:12.961 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-16T22:15:12.963 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-16T22:15:12.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T22:15:13.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T22:15:13.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:13.252+0000 7f7b35ca5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T22:15:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:13.252+0000 7f7b35ca5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T22:15:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:13.252+0000 7f7b35ca5640 -1 bdev(0x562ea2697800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:13.253+0000 7f7b35ca5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T22:15:16.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T22:15:16.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:16.292+0000 7f51b1545640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T22:15:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:16.292+0000 7f51b1545640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T22:15:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:16.292+0000 7f51b1545640 -1 bdev(0x55649f32b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:16.292+0000 7f51b1545640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T22:15:19.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T22:15:19.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:19.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:19.836+0000 7feae9483640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T22:15:19.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:19.837+0000 7feae9483640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T22:15:19.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:19.837+0000 7feae9483640 -1 bdev(0x5559fd7cd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:19.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:19.837+0000 7feae9483640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T22:15:23.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T22:15:23.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:23.364+0000 7ff0ebab4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T22:15:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:23.365+0000 7ff0ebab4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T22:15:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:23.365+0000 7ff0ebab4640 -1 bdev(0x55664ff7b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:15:23.365+0000 7ff0ebab4640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T22:15:26.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T22:15:26.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T22:15:26.329 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:26.329 INFO:tasks.ceph:role: osd.4 2024-09-16T22:15:26.329 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:15:26.329 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:26.404 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:26.405 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:26.405 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:26.914 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-16T22:15:26.917 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-16T22:15:26.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T22:15:27.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T22:15:27.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T22:15:27.136 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:27.137 INFO:tasks.ceph:role: osd.5 2024-09-16T22:15:27.137 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:15:27.137 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T22:15:27.209 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:27.209 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:27.209 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:27.209 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:27.210 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-16T22:15:27.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-16T22:15:27.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T22:15:27.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T22:15:27.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T22:15:27.867 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:27.868 INFO:tasks.ceph:role: osd.6 2024-09-16T22:15:27.868 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:15:27.868 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:27.939 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:27.940 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:28.421 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-16T22:15:28.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-16T22:15:28.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T22:15:28.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T22:15:28.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T22:15:28.660 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T22:15:28.660 INFO:tasks.ceph:role: osd.7 2024-09-16T22:15:28.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:15:28.660 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-16T22:15:28.732 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T22:15:28.733 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T22:15:28.733 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T22:15:28.733 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T22:15:29.211 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-16T22:15:29.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-16T22:15:29.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T22:15:29.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T22:15:29.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:29.517+0000 7ff1f47b2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T22:15:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:29.517+0000 7ff1f47b2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T22:15:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:29.517+0000 7ff1f47b2640 -1 bdev(0x558b62d8d800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:29.517+0000 7ff1f47b2640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T22:15:32.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T22:15:32.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:32.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:32.543+0000 7f9a65bae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T22:15:32.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:32.543+0000 7f9a65bae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T22:15:32.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:32.543+0000 7f9a65bae640 -1 bdev(0x556a92061800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:32.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:32.544+0000 7f9a65bae640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T22:15:35.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T22:15:35.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:35.812+0000 7fe572f34640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T22:15:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:35.813+0000 7fe572f34640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T22:15:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:35.813+0000 7fe572f34640 -1 bdev(0x5564b020d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:35.813+0000 7fe572f34640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T22:15:38.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T22:15:39.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:39.101+0000 7f6a60460640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T22:15:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:39.101+0000 7f6a60460640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T22:15:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:39.101+0000 7f6a60460640 -1 bdev(0x55f13a051800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-16T22:15:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-16T22:15:39.101+0000 7f6a60460640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T22:15:42.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T22:15:42.032 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T22:15:42.032 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T22:15:42.064 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T22:15:42.135 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T22:15:42.205 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T22:15:42.275 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T22:15:42.344 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T22:15:42.373 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T22:15:42.442 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T22:15:42.512 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T22:15:42.581 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T22:15:42.650 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:42.650 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T22:15:42.670 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:42.670 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T22:15:42.708 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T22:15:42.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T22:15:42.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T22:15:42.752 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-09-16T22:15:42.752 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDtrehmtNWoMRAAQd/eU4Gn9zi8qH3XIDcGLA== 2024-09-16T22:15:42.752 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-16T22:15:42.752 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxrehmEBUcDxAA7mCHv+vtnGIHKd4312HATw== 2024-09-16T22:15:42.752 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD0rehmDuV8ERAAIA5/3ObJVaFQ3prD5zuEeg== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD3rehm0BbxMRAAqp3twyEP7OOIxceDdH5EyA== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD7rehmLG7KFRAAnzWvy5zr+ZJpzvVkhYc+Yw== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDtrehmnK65NhAA7v9hNMMRmH+dUypjI33M/g== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQABruhm5OXfHhAAkNIOwKlRC27xTkgRY+jQKw== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAEruhmMplzIBAAKvjrBJqzPCR2oCCwRfZV4Q== 2024-09-16T22:15:42.753 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAHruhmiJCNMBAAS0gCafCBwfPEqr1DHVM9Kw== 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQALruhm3JgVBhAA2RcuVSIyJN+kYdsjpeJmDg== 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDtrehmeq13ORAADUgXsfng32JpSszsgYvA9g== 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-09-16T22:15:42.754 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDurehmA+sZARAA07T2WyTIcNNjxsLmfUVTPw== 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDtrehmtNWoMRAAQd/eU4Gn9zi8qH3XIDcGLA== 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDxrehmEBUcDxAA7mCHv+vtnGIHKd4312HATw== 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD0rehmDuV8ERAAIA5/3ObJVaFQ3prD5zuEeg== 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-09-16T22:15:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD3rehm0BbxMRAAqp3twyEP7OOIxceDdH5EyA== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD7rehmLG7KFRAAnzWvy5zr+ZJpzvVkhYc+Yw== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDtrehmnK65NhAA7v9hNMMRmH+dUypjI33M/g== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQABruhm5OXfHhAAkNIOwKlRC27xTkgRY+jQKw== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAEruhmMplzIBAAKvjrBJqzPCR2oCCwRfZV4Q== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAHruhmiJCNMBAAS0gCafCBwfPEqr1DHVM9Kw== 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-09-16T22:15:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQALruhm3JgVBhAA2RcuVSIyJN+kYdsjpeJmDg== 2024-09-16T22:15:42.778 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-16T22:15:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDtrehmeq13ORAADUgXsfng32JpSszsgYvA9g== 2024-09-16T22:15:42.778 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2024-09-16T22:15:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDurehmA+sZARAA07T2WyTIcNNjxsLmfUVTPw== 2024-09-16T22:15:42.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T22:15:42.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T22:15:42.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:42.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:42.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:42.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:42.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:42.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T22:15:43.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T22:15:43.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T22:15:43.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T22:15:43.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T22:15:43.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T22:15:43.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T22:15:43.476 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T22:15:43.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T22:15:43.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T22:15:43.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T22:15:43.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T22:15:43.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T22:15:43.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T22:15:43.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T22:15:43.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T22:15:43.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T22:15:43.962 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:43.964 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T22:15:44.017 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T22:15:44.017 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T22:15:44.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T22:15:44.020 INFO:tasks.ceph.mon.a:Started 2024-09-16T22:15:44.020 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T22:15:44.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T22:15:44.022 INFO:tasks.ceph.mon.c:Started 2024-09-16T22:15:44.022 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T22:15:44.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T22:15:44.060 INFO:tasks.ceph.mon.b:Started 2024-09-16T22:15:44.060 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T22:15:44.060 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T22:15:44.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T22:15:44.062 INFO:tasks.ceph.mgr.y:Started 2024-09-16T22:15:44.062 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T22:15:44.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T22:15:44.065 INFO:tasks.ceph.mgr.x:Started 2024-09-16T22:15:44.065 DEBUG:tasks.ceph:set 0 configs 2024-09-16T22:15:44.065 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T22:15:44.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T22:15:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-16T22:15:44.424 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T22:15:44.424 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T22:15:44.425 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:44.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T22:15:44.450 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:44.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T22:15:44.515 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:44.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T22:15:44.581 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T22:15:44.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T22:15:44.647 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:44.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T22:15:44.672 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:44.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T22:15:44.737 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:44.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T22:15:44.802 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-16T22:15:44.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T22:15:44.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 44de8de0-e16a-4325-94b9-1cc61f4829db 0 2024-09-16T22:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout:0 2024-09-16T22:15:45.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 0e05fbd1-9707-40b8-91a3-198c6f5b7865 1 2024-09-16T22:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-16T22:15:45.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new b080bc29-0a92-4200-ae3f-0a17a1722145 2 2024-09-16T22:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout:2 2024-09-16T22:15:45.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 0bc64c75-486c-40ac-abaf-381e151623bd 3 2024-09-16T22:15:45.669 INFO:teuthology.orchestra.run.smithi092.stdout:3 2024-09-16T22:15:45.680 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new fae627d1-6bc2-4e5b-85e1-334e5e05f830 4 2024-09-16T22:15:45.866 INFO:teuthology.orchestra.run.smithi092.stdout:4 2024-09-16T22:15:45.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 989a79cd-ef84-4bf3-a64f-9d101d0b29d7 5 2024-09-16T22:15:46.058 INFO:teuthology.orchestra.run.smithi092.stdout:5 2024-09-16T22:15:46.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 92f145b7-e868-43fd-9abe-68c43a407ae2 6 2024-09-16T22:15:46.248 INFO:teuthology.orchestra.run.smithi092.stdout:6 2024-09-16T22:15:46.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 373f8002-53a4-4ca7-bbbf-70e8d56b3571 7 2024-09-16T22:15:46.438 INFO:teuthology.orchestra.run.smithi092.stdout:7 2024-09-16T22:15:46.450 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T22:15:46.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T22:15:46.453 INFO:tasks.ceph.osd.0:Started 2024-09-16T22:15:46.453 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T22:15:46.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T22:15:46.455 INFO:tasks.ceph.osd.1:Started 2024-09-16T22:15:46.455 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T22:15:46.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T22:15:46.458 INFO:tasks.ceph.osd.2:Started 2024-09-16T22:15:46.458 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T22:15:46.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T22:15:46.460 INFO:tasks.ceph.osd.3:Started 2024-09-16T22:15:46.461 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T22:15:46.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T22:15:46.463 INFO:tasks.ceph.osd.4:Started 2024-09-16T22:15:46.463 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T22:15:46.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T22:15:46.465 INFO:tasks.ceph.osd.5:Started 2024-09-16T22:15:46.465 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T22:15:46.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T22:15:46.467 INFO:tasks.ceph.osd.6:Started 2024-09-16T22:15:46.468 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T22:15:46.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T22:15:46.470 INFO:tasks.ceph.osd.7:Started 2024-09-16T22:15:46.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:15:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:15:46.661 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:15:46.438697+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:15:46.672 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T22:15:46.672 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T22:15:47.438 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:15:47.439+0000 7f5f771b2640 -1 Falling back to public interface 2024-09-16T22:15:47.478 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T22:15:47.644 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-16T22:15:47.644 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:15:46.438697+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:15:47.653 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T22:15:47.681 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:15:47.682+0000 7f44f75b0640 -1 Falling back to public interface 2024-09-16T22:15:47.691 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:15:47.692+0000 7f49e960d640 -1 Falling back to public interface 2024-09-16T22:15:47.905 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:15:47.906+0000 7f2873885640 -1 Falling back to public interface 2024-09-16T22:15:47.939 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:15:47.940+0000 7f7e6ec1c640 -1 Falling back to public interface 2024-09-16T22:15:48.170 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:15:48.171+0000 7f46c21b2640 -1 Falling back to public interface 2024-09-16T22:15:48.174 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:15:48.175+0000 7f479ef1d640 -1 Falling back to public interface 2024-09-16T22:15:48.193 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:15:48.195+0000 7f29ec72b640 -1 Falling back to public interface 2024-09-16T22:15:49.570 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-09-16T22:15:49.571+0000 7fa03c6ae640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T22:15:49.570 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-09-16T22:15:49.571+0000 7fa03c6ae640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T22:15:51.488 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:15:51.490+0000 7f49e960d640 -1 osd.7 0 log_to_monitors true 2024-09-16T22:15:51.721 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:15:51.722+0000 7f7e6ec1c640 -1 osd.6 0 log_to_monitors true 2024-09-16T22:15:51.730 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:15:51.731+0000 7f44f75b0640 -1 osd.2 0 log_to_monitors true 2024-09-16T22:15:51.739 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:15:51.740+0000 7f29ec72b640 -1 osd.4 0 log_to_monitors true 2024-09-16T22:15:51.754 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:15:51.755+0000 7f5f771b2640 -1 osd.1 0 log_to_monitors true 2024-09-16T22:15:51.989 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:15:51.990+0000 7f479ef1d640 -1 osd.3 0 log_to_monitors true 2024-09-16T22:15:52.002 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:15:52.004+0000 7f2873885640 -1 osd.0 0 log_to_monitors true 2024-09-16T22:15:52.181 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:15:52.182+0000 7f46c21b2640 -1 osd.5 0 log_to_monitors true 2024-09-16T22:15:53.476 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:15:53.477+0000 7f49e585f640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T22:15:53.490 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:15:53.492+0000 7f49dce6a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:54.460 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T22:15:54.796 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-16T22:15:54.796 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":13,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:15:53.581044+0000","last_up_change":"2024-09-16T22:15:53.581044+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:15:54.808 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-16T22:15:55.365 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:15:55.366+0000 7f46be141640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T22:15:55.371 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:15:55.372+0000 7f7e6adb5640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T22:15:55.378 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:15:55.380+0000 7f46b5f60640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.388 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:15:55.390+0000 7f7e623c0640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.398 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:15:55.399+0000 7f29e8ece640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T22:15:55.412 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:15:55.414+0000 7f29e04d9640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.841 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:15:55.841+0000 7f286f814640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T22:15:55.857 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:15:55.857+0000 7f44f353f640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T22:15:55.861 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:15:55.862+0000 7f2867633640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.867 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:15:55.867+0000 7f5f73955640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T22:15:55.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:15:55.876+0000 7f44eb35e640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.886 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:15:55.886+0000 7f479b6c0640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T22:15:55.887 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:15:55.888+0000 7f5f6af60640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:15:55.907 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:15:55.909+0000 7f4792ccb640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T22:16:01.616 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T22:16:01.919 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-16T22:16:01.919 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":18,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:15:59.383685+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:16:01.930 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T22:16:01.930 INFO:tasks.ceph:Creating RBD pool 2024-09-16T22:16:01.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T22:16:02.413 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-09-16T22:16:02.426 DEBUG:teuthology.orchestra.run.smithi031:> rbd --cluster ceph pool init rbd 2024-09-16T22:16:05.434 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T22:16:05.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T22:16:05.435 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T22:16:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:05.759 INFO:teuthology.orchestra.run.smithi031.stdout:[{"version":9,"timestamp":"2024-09-16T22:15:55.889204+0000","name":"","changes":[{"name":"osd.3/osd_mclock_max_capacity_iops_ssd","new_value":"3971.962562"}]}] 2024-09-16T22:16:05.760 INFO:tasks.ceph_manager:config epoch is 9 2024-09-16T22:16:05.760 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T22:16:05.760 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T22:16:05.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T22:16:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:06.123 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3024744632},{"type":"v1","addr":"172.21.15.92:6833","nonce":3024744632}]},"active_addr":"172.21.15.92:6833/3024744632","active_change":"2024-09-16T22:15:48.560006+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":877571319}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":875207962}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1962007859}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2554100272}]}]} 2024-09-16T22:16:06.127 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T22:16:06.127 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T22:16:06.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:16:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:06.426 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:16:05.419708+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T22:16:02.234097+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:16:06.439 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T22:16:06.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:16:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:06.742 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:16:05.419708+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T22:16:02.234097+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:16:06.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T22:16:06.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T22:16:06.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T22:16:06.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T22:16:06.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T22:16:06.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T22:16:06.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T22:16:06.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T22:16:07.008 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:07.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T22:16:07.009 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:07.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T22:16:07.010 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:07.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T22:16:07.012 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:07.012 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:07.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T22:16:07.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T22:16:07.014 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:07.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T22:16:07.015 INFO:teuthology.orchestra.run.smithi031.stdout:64424509447 2024-09-16T22:16:07.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T22:16:07.019 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2024-09-16T22:16:07.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T22:16:07.605 INFO:teuthology.orchestra.run.smithi031.stdout:64424509443 2024-09-16T22:16:07.611 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:07.621 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:07.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.0 2024-09-16T22:16:07.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.6 2024-09-16T22:16:07.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.4 2024-09-16T22:16:07.655 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-09-16T22:16:07.663 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:07.663 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:07.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-16T22:16:07.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.1 2024-09-16T22:16:07.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.2 2024-09-16T22:16:07.687 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:07.697 INFO:teuthology.orchestra.run.smithi031.stdout:64424509443 2024-09-16T22:16:07.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2024-09-16T22:16:07.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.3 2024-09-16T22:16:08.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T22:16:08.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T22:16:08.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T22:16:08.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T22:16:08.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T22:16:08.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T22:16:08.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T22:16:08.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T22:16:09.211 INFO:teuthology.orchestra.run.smithi031.stdout:64424509443 2024-09-16T22:16:09.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.0 2024-09-16T22:16:09.258 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-09-16T22:16:09.269 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:09.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-16T22:16:09.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.6 2024-09-16T22:16:09.286 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:09.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.4 2024-09-16T22:16:09.308 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:09.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.1 2024-09-16T22:16:09.329 INFO:teuthology.orchestra.run.smithi031.stdout:64424509443 2024-09-16T22:16:09.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.3 2024-09-16T22:16:09.356 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:09.356 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-16T22:16:09.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2024-09-16T22:16:09.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.2 2024-09-16T22:16:10.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T22:16:10.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T22:16:10.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T22:16:10.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T22:16:10.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T22:16:10.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T22:16:10.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T22:16:10.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T22:16:10.797 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:10.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-09-16T22:16:10.816 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.839 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2024-09-16T22:16:10.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2024-09-16T22:16:10.858 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.861 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:10.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-16T22:16:10.875 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.923 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:10.930 INFO:teuthology.orchestra.run.smithi031.stdout:64424509447 2024-09-16T22:16:10.932 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:10.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-09-16T22:16:10.941 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-16T22:16:10.947 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-16T22:16:10.951 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.962 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-16T22:16:10.963 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-16T22:16:10.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-16T22:16:10.981 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-16T22:16:10.982 DEBUG:teuthology.parallel:result is None 2024-09-16T22:16:10.982 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T22:16:10.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:11.317 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:11.328 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T22:16:10.568943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":2965504,"data_stored":1776918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.149388"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430191+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:05.430191+0000","last_peered":"2024-09-16T22:16:05.430191+0000","last_clean":"2024-09-16T22:16:05.430191+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:05.430191+0000","last_undegraded":"2024-09-16T22:16:05.430191+0000","last_fullsized":"2024-09-16T22:16:05.430191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.824266+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:05.824266+0000","last_peered":"2024-09-16T22:16:05.824266+0000","last_clean":"2024-09-16T22:16:05.824266+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:05.824266+0000","last_undegraded":"2024-09-16T22:16:05.824266+0000","last_fullsized":"2024-09-16T22:16:05.824266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430483+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:05.430483+0000","last_peered":"2024-09-16T22:16:05.430483+0000","last_clean":"2024-09-16T22:16:05.430483+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:05.430483+0000","last_undegraded":"2024-09-16T22:16:05.430483+0000","last_fullsized":"2024-09-16T22:16:05.430483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426817+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:05.426817+0000","last_peered":"2024-09-16T22:16:05.426817+0000","last_clean":"2024-09-16T22:16:05.426817+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:05.426817+0000","last_undegraded":"2024-09-16T22:16:05.426817+0000","last_fullsized":"2024-09-16T22:16:05.426817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430960+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:05.430960+0000","last_peered":"2024-09-16T22:16:05.430960+0000","last_clean":"2024-09-16T22:16:05.430960+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:05.430960+0000","last_undegraded":"2024-09-16T22:16:05.430960+0000","last_fullsized":"2024-09-16T22:16:05.430960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.930317+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:05.930317+0000","last_peered":"2024-09-16T22:16:05.930317+0000","last_clean":"2024-09-16T22:16:05.930317+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:05.930317+0000","last_undegraded":"2024-09-16T22:16:05.930317+0000","last_fullsized":"2024-09-16T22:16:05.930317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.429704+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:05.429704+0000","last_peered":"2024-09-16T22:16:05.429704+0000","last_clean":"2024-09-16T22:16:05.429704+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:05.429704+0000","last_undegraded":"2024-09-16T22:16:05.429704+0000","last_fullsized":"2024-09-16T22:16:05.429704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430542+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:05.430542+0000","last_peered":"2024-09-16T22:16:05.430542+0000","last_clean":"2024-09-16T22:16:05.430542+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:05.430542+0000","last_undegraded":"2024-09-16T22:16:05.430542+0000","last_fullsized":"2024-09-16T22:16:05.430542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426110+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:05.426110+0000","last_peered":"2024-09-16T22:16:05.426110+0000","last_clean":"2024-09-16T22:16:05.426110+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:05.426110+0000","last_undegraded":"2024-09-16T22:16:05.426110+0000","last_fullsized":"2024-09-16T22:16:05.426110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:11.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:11.622 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:11.635 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T22:16:10.568943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":2965504,"data_stored":1776918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.149388"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430191+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:05.430191+0000","last_peered":"2024-09-16T22:16:05.430191+0000","last_clean":"2024-09-16T22:16:05.430191+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:05.430191+0000","last_undegraded":"2024-09-16T22:16:05.430191+0000","last_fullsized":"2024-09-16T22:16:05.430191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.824266+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:05.824266+0000","last_peered":"2024-09-16T22:16:05.824266+0000","last_clean":"2024-09-16T22:16:05.824266+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:05.824266+0000","last_undegraded":"2024-09-16T22:16:05.824266+0000","last_fullsized":"2024-09-16T22:16:05.824266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430483+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:05.430483+0000","last_peered":"2024-09-16T22:16:05.430483+0000","last_clean":"2024-09-16T22:16:05.430483+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:05.430483+0000","last_undegraded":"2024-09-16T22:16:05.430483+0000","last_fullsized":"2024-09-16T22:16:05.430483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426817+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:05.426817+0000","last_peered":"2024-09-16T22:16:05.426817+0000","last_clean":"2024-09-16T22:16:05.426817+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:05.426817+0000","last_undegraded":"2024-09-16T22:16:05.426817+0000","last_fullsized":"2024-09-16T22:16:05.426817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430960+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:05.430960+0000","last_peered":"2024-09-16T22:16:05.430960+0000","last_clean":"2024-09-16T22:16:05.430960+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:05.430960+0000","last_undegraded":"2024-09-16T22:16:05.430960+0000","last_fullsized":"2024-09-16T22:16:05.430960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.930317+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:05.930317+0000","last_peered":"2024-09-16T22:16:05.930317+0000","last_clean":"2024-09-16T22:16:05.930317+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:05.930317+0000","last_undegraded":"2024-09-16T22:16:05.930317+0000","last_fullsized":"2024-09-16T22:16:05.930317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.429704+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:05.429704+0000","last_peered":"2024-09-16T22:16:05.429704+0000","last_clean":"2024-09-16T22:16:05.429704+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:05.429704+0000","last_undegraded":"2024-09-16T22:16:05.429704+0000","last_fullsized":"2024-09-16T22:16:05.429704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430542+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:05.430542+0000","last_peered":"2024-09-16T22:16:05.430542+0000","last_clean":"2024-09-16T22:16:05.430542+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:05.430542+0000","last_undegraded":"2024-09-16T22:16:05.430542+0000","last_fullsized":"2024-09-16T22:16:05.430542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426110+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:05.426110+0000","last_peered":"2024-09-16T22:16:05.426110+0000","last_clean":"2024-09-16T22:16:05.426110+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:05.426110+0000","last_undegraded":"2024-09-16T22:16:05.426110+0000","last_fullsized":"2024-09-16T22:16:05.426110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:11.635 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T22:16:11.635 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T22:16:11.636 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T22:16:11.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T22:16:11.969 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:11.969 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T22:16:11.981 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T22:16:11.981 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T22:16:11.992 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T22:16:11.992 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-16T22:16:11.992 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T22:16:11.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T22:16:12.312 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T22:16:12.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T22:16:12.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:12.619 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T22:16:12.569365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":2965504,"data_stored":1776918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.152836"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430191+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:05.430191+0000","last_peered":"2024-09-16T22:16:05.430191+0000","last_clean":"2024-09-16T22:16:05.430191+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:05.430191+0000","last_undegraded":"2024-09-16T22:16:05.430191+0000","last_fullsized":"2024-09-16T22:16:05.430191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.824266+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:05.824266+0000","last_peered":"2024-09-16T22:16:05.824266+0000","last_clean":"2024-09-16T22:16:05.824266+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:05.824266+0000","last_undegraded":"2024-09-16T22:16:05.824266+0000","last_fullsized":"2024-09-16T22:16:05.824266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430483+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:05.430483+0000","last_peered":"2024-09-16T22:16:05.430483+0000","last_clean":"2024-09-16T22:16:05.430483+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:05.430483+0000","last_undegraded":"2024-09-16T22:16:05.430483+0000","last_fullsized":"2024-09-16T22:16:05.430483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426817+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:05.426817+0000","last_peered":"2024-09-16T22:16:05.426817+0000","last_clean":"2024-09-16T22:16:05.426817+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:05.426817+0000","last_undegraded":"2024-09-16T22:16:05.426817+0000","last_fullsized":"2024-09-16T22:16:05.426817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430960+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:05.430960+0000","last_peered":"2024-09-16T22:16:05.430960+0000","last_clean":"2024-09-16T22:16:05.430960+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:05.430960+0000","last_undegraded":"2024-09-16T22:16:05.430960+0000","last_fullsized":"2024-09-16T22:16:05.430960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.930317+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:05.930317+0000","last_peered":"2024-09-16T22:16:05.930317+0000","last_clean":"2024-09-16T22:16:05.930317+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:05.930317+0000","last_undegraded":"2024-09-16T22:16:05.930317+0000","last_fullsized":"2024-09-16T22:16:05.930317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.429704+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:05.429704+0000","last_peered":"2024-09-16T22:16:05.429704+0000","last_clean":"2024-09-16T22:16:05.429704+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:05.429704+0000","last_undegraded":"2024-09-16T22:16:05.429704+0000","last_fullsized":"2024-09-16T22:16:05.429704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430542+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:05.430542+0000","last_peered":"2024-09-16T22:16:05.430542+0000","last_clean":"2024-09-16T22:16:05.430542+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:05.430542+0000","last_undegraded":"2024-09-16T22:16:05.430542+0000","last_fullsized":"2024-09-16T22:16:05.430542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426110+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:05.426110+0000","last_peered":"2024-09-16T22:16:05.426110+0000","last_clean":"2024-09-16T22:16:05.426110+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:05.426110+0000","last_undegraded":"2024-09-16T22:16:05.426110+0000","last_fullsized":"2024-09-16T22:16:05.426110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:12.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:12.916 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:12.917 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:12.928 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T22:16:12.569365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":2965504,"data_stored":1776918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.152836"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430191+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:05.430191+0000","last_peered":"2024-09-16T22:16:05.430191+0000","last_clean":"2024-09-16T22:16:05.430191+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:05.430191+0000","last_undegraded":"2024-09-16T22:16:05.430191+0000","last_fullsized":"2024-09-16T22:16:05.430191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.824266+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:05.824266+0000","last_peered":"2024-09-16T22:16:05.824266+0000","last_clean":"2024-09-16T22:16:05.824266+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:05.824266+0000","last_undegraded":"2024-09-16T22:16:05.824266+0000","last_fullsized":"2024-09-16T22:16:05.824266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430483+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:05.430483+0000","last_peered":"2024-09-16T22:16:05.430483+0000","last_clean":"2024-09-16T22:16:05.430483+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:05.430483+0000","last_undegraded":"2024-09-16T22:16:05.430483+0000","last_fullsized":"2024-09-16T22:16:05.430483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426817+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:05.426817+0000","last_peered":"2024-09-16T22:16:05.426817+0000","last_clean":"2024-09-16T22:16:05.426817+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:05.426817+0000","last_undegraded":"2024-09-16T22:16:05.426817+0000","last_fullsized":"2024-09-16T22:16:05.426817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430960+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:05.430960+0000","last_peered":"2024-09-16T22:16:05.430960+0000","last_clean":"2024-09-16T22:16:05.430960+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:05.430960+0000","last_undegraded":"2024-09-16T22:16:05.430960+0000","last_fullsized":"2024-09-16T22:16:05.430960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.930317+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:05.930317+0000","last_peered":"2024-09-16T22:16:05.930317+0000","last_clean":"2024-09-16T22:16:05.930317+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:05.930317+0000","last_undegraded":"2024-09-16T22:16:05.930317+0000","last_fullsized":"2024-09-16T22:16:05.930317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.429704+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:05.429704+0000","last_peered":"2024-09-16T22:16:05.429704+0000","last_clean":"2024-09-16T22:16:05.429704+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:05.429704+0000","last_undegraded":"2024-09-16T22:16:05.429704+0000","last_fullsized":"2024-09-16T22:16:05.429704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430542+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:05.430542+0000","last_peered":"2024-09-16T22:16:05.430542+0000","last_clean":"2024-09-16T22:16:05.430542+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:05.430542+0000","last_undegraded":"2024-09-16T22:16:05.430542+0000","last_fullsized":"2024-09-16T22:16:05.430542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426110+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:05.426110+0000","last_peered":"2024-09-16T22:16:05.426110+0000","last_clean":"2024-09-16T22:16:05.426110+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:05.426110+0000","last_undegraded":"2024-09-16T22:16:05.426110+0000","last_fullsized":"2024-09-16T22:16:05.426110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:12.929 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T22:16:12.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 22 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 57fe7da5-e356-48a4-ba33-5558b1026a32 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T22:15:44.319460+0000 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T22:16:05.419708+0000 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T22:16:13.226 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 5 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-16T22:16:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: removed_snaps_queue [2~1] 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 8 2024-09-16T22:16:13.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/2585124124,v1:172.21.15.31:6817/2585124124] [v2:172.21.15.31:6818/2585124124,v1:172.21.15.31:6819/2585124124] exists,up 44de8de0-e16a-4325-94b9-1cc61f4829db 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/3271285223,v1:172.21.15.31:6801/3271285223] [v2:172.21.15.31:6802/3271285223,v1:172.21.15.31:6803/3271285223] exists,up 0e05fbd1-9707-40b8-91a3-198c6f5b7865 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2874892035,v1:172.21.15.31:6809/2874892035] [v2:172.21.15.31:6810/2874892035,v1:172.21.15.31:6811/2874892035] exists,up b080bc29-0a92-4200-ae3f-0a17a1722145 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/3768411773,v1:172.21.15.31:6825/3768411773] [v2:172.21.15.31:6826/3768411773,v1:172.21.15.31:6827/3768411773] exists,up 0bc64c75-486c-40ac-abaf-381e151623bd 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/3421901083,v1:172.21.15.92:6825/3421901083] [v2:172.21.15.92:6826/3421901083,v1:172.21.15.92:6827/3421901083] exists,up fae627d1-6bc2-4e5b-85e1-334e5e05f830 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/335833224,v1:172.21.15.92:6817/335833224] [v2:172.21.15.92:6818/335833224,v1:172.21.15.92:6819/335833224] exists,up 989a79cd-ef84-4bf3-a64f-9d101d0b29d7 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1592419018,v1:172.21.15.92:6809/1592419018] [v2:172.21.15.92:6810/1592419018,v1:172.21.15.92:6811/1592419018] exists,up 92f145b7-e868-43fd-9abe-68c43a407ae2 2024-09-16T22:16:13.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3448871901,v1:172.21.15.92:6801/3448871901] [v2:172.21.15.92:6802/3448871901,v1:172.21.15.92:6803/3448871901] exists,up 373f8002-53a4-4ca7-bbbf-70e8d56b3571 2024-09-16T22:16:13.239 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/2585124124,v1:172.21.15.31:6817/2585124124] [v2:172.21.15.31:6818/2585124124,v1:172.21.15.31:6819/2585124124] exists,up 44de8de0-e16a-4325-94b9-1cc61f4829db', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/3271285223,v1:172.21.15.31:6801/3271285223] [v2:172.21.15.31:6802/3271285223,v1:172.21.15.31:6803/3271285223] exists,up 0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2874892035,v1:172.21.15.31:6809/2874892035] [v2:172.21.15.31:6810/2874892035,v1:172.21.15.31:6811/2874892035] exists,up b080bc29-0a92-4200-ae3f-0a17a1722145', 'osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/3768411773,v1:172.21.15.31:6825/3768411773] [v2:172.21.15.31:6826/3768411773,v1:172.21.15.31:6827/3768411773] exists,up 0bc64c75-486c-40ac-abaf-381e151623bd', 'osd.4 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/3421901083,v1:172.21.15.92:6825/3421901083] [v2:172.21.15.92:6826/3421901083,v1:172.21.15.92:6827/3421901083] exists,up fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/335833224,v1:172.21.15.92:6817/335833224] [v2:172.21.15.92:6818/335833224,v1:172.21.15.92:6819/335833224] exists,up 989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1592419018,v1:172.21.15.92:6809/1592419018] [v2:172.21.15.92:6810/1592419018,v1:172.21.15.92:6811/1592419018] exists,up 92f145b7-e868-43fd-9abe-68c43a407ae2', 'osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3448871901,v1:172.21.15.92:6801/3448871901] [v2:172.21.15.92:6802/3448871901,v1:172.21.15.92:6803/3448871901] exists,up 373f8002-53a4-4ca7-bbbf-70e8d56b3571'] 2024-09-16T22:16:13.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-09-16T22:16:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard_threads": "0", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T22:16:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T22:16:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T22:16:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T22:16:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi": "true", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T22:16:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T22:16:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T22:16:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T22:16:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T22:16:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T22:16:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T22:16:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-09-16T22:16:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T22:16:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T22:16:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T22:16:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T22:16:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T22:16:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T22:16:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-09-16T22:16:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T22:16:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_write_v2": "false", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_write_v2_random": "false", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T22:16:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-09-16T22:16:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_features": "", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T22:16:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-09-16T22:16:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-09-16T22:16:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-09-16T22:16:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T22:16:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_stat_interval": "0", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_cpu_cores": "", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_num_threads": "0", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T22:16:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-09-16T22:16:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_dedup": "0/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crimson_interrupt": "0/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-09-16T22:16:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T22:16:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T22:16:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_quiesce": "3/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-09-16T22:16:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-09-16T22:16:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_notification": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-09-16T22:16:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-09-16T22:16:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_trace": "1/5", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T22:16:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_cert_file": "", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_key_file": "", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-09-16T22:16:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-09-16T22:16:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T22:16:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T22:16:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T22:16:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T22:16:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T22:16:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-09-16T22:16:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T22:16:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-09-16T22:16:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T22:16:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-09-16T22:16:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-09-16T22:16:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-09-16T22:16:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T22:16:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-09-16T22:16:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_overload_epochs": "2", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T22:16:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_delay": "0", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T22:16:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-09-16T22:16:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T22:16:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T22:16:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_dirfrag_at": "0", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-09-16T22:16:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_shutdown_at": "0", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_event_large_threshold": "524288", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-09-16T22:16:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_threshold": "128", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T22:16:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-09-16T22:16:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T22:16:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scrub_stats_review_period": "1", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T22:16:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T22:16:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-09-16T22:16:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_enable_op_tracker": "true", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_creating": "1024", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T22:16:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_duration": "600", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_size": "20", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_log_threshold": "5", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T22:16:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T22:16:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_target_rank": "-1", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_level": "debug", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T22:16:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T22:16:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-09-16T22:16:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_squid": "false", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_uptime_grace": "60", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T22:16:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.92", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T22:16:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T22:16:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T22:16:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T22:16:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T22:16:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T22:16:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T22:16:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-09-16T22:16:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T22:16:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T22:16:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T22:16:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T22:16:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-09-16T22:16:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-09-16T22:16:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T22:16:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-09-16T22:16:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T22:16:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T22:16:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-16T22:16:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T22:16:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T22:16:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-09-16T22:16:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T22:16:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-09-16T22:16:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_trim_objects": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T22:16:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-09-16T22:16:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ec_partial_reads": "true", 2024-09-16T22:16:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-09-16T22:16:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T22:16:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T22:16:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-09-16T22:16:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T22:16:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T22:16:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-09-16T22:16:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T22:16:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-09-16T22:16:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T22:16:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T22:16:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_ratio": "70", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T22:16:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T22:16:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T22:16:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T22:16:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_retry_delay": "30", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_retry_trimming": "10", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-16T22:16:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T22:16:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-09-16T22:16:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-09-16T22:16:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_busy_polling": "false", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-09-16T22:16:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T22:16:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T22:16:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-09-16T22:16:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T22:16:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T22:16:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T22:16:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T22:16:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T22:16:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-09-16T22:16:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_asio_assert_yielding": "false", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache": "false", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T22:16:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "25000", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T22:16:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T22:16:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T22:16:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T22:16:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-16T22:16:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T22:16:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_disable_s3select": "false", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T22:16:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-16T22:16:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_mdsearch": "true", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-09-16T22:16:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T22:16:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-09-16T22:16:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T22:16:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T22:16:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T22:16:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T22:16:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "5000", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T22:16:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T22:16:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T22:16:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-09-16T22:16:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lanes": "3", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-16T22:16:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_partitions": "3", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T22:16:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T22:16:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T22:16:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T22:16:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T22:16:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-16T22:16:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-09-16T22:16:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache": "false", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T22:16:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T22:16:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T22:16:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-09-16T22:16:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_full_integrity_check": "false", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T22:16:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T22:16:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tmp_dir": "/tmp", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uadk_compressor_enabled": "false", 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-16T22:16:13.505 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:13.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T22:16:13.623 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-16T22:16:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:16:13.633 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-16T22:16:13.633 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:16:13.642 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-16T22:16:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:16:13.653 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-09-16T22:16:13.671 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T22:16:13.671 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-09-16T22:16:13.686 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T22:16:13.686 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T22:16:13.701 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T22:16:13.702 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T22:16:13.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:13.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:13.702 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T22:16:13.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T22:16:13.703 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T22:16:13.703 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T22:16:13.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:13.703 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T22:16:13.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:13.704 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T22:16:13.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:16:13.705 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-16T22:16:13.712 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:13.714+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:13.769 INFO:teuthology.orchestra.run.smithi031.stdout:mapped writes DISABLED 2024-09-16T22:16:13.769 INFO:teuthology.orchestra.run.smithi031.stdout:mapped reads DISABLED 2024-09-16T22:16:13.769 INFO:teuthology.orchestra.run.smithi031.stdout:Seed set to 8924 2024-09-16T22:16:13.810 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:13.812+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:13.865 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:13.866 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:13.866 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:13.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:13.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:13.886 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:13.904 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:13.904 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:13.905 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:13.907+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:13.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:13.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:14.007 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:14.007 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:14.007+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:14.042 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:14.107 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:14.109+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:16:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 23, 2024-09-16T22:16:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-16T22:15:44.319460+0000", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-16T22:16:14.110173+0000", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-16T22:15:56.374983+0000", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-16T22:15:46.438697+0000", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-16T22:16:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:15:56.583719+0000", 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-16T22:16:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:02.234097+0000", 2024-09-16T22:16:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "22", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 22, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-16T22:16:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:13.789572+0000", 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 32769, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,creating", 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "23", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": {}, 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:16:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "44de8de0-e16a-4325-94b9-1cc61f4829db", 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-16T22:16:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-16T22:16:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2585124124", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2585124124", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2585124124", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2585124124", 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0e05fbd1-9707-40b8-91a3-198c6f5b7865", 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 16, 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T22:16:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-16T22:16:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-16T22:16:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3271285223", 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3271285223", 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3271285223", 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3271285223", 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "b080bc29-0a92-4200-ae3f-0a17a1722145", 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-16T22:16:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-16T22:16:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2874892035", 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2874892035", 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2874892035", 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2874892035", 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0bc64c75-486c-40ac-abaf-381e151623bd", 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-16T22:16:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-16T22:16:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3768411773", 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3768411773", 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3768411773", 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3768411773", 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fae627d1-6bc2-4e5b-85e1-334e5e05f830", 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6824", 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6825", 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6826", 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6827", 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6830", 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6831", 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6828", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6829", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6825/3421901083", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6827/3421901083", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3421901083", 2024-09-16T22:16:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6829/3421901083", 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "989a79cd-ef84-4bf3-a64f-9d101d0b29d7", 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6816", 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6817", 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6818", 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6819", 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6822", 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6823", 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6820", 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6821", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6817/335833224", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6819/335833224", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6823/335833224", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6821/335833224", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "92f145b7-e868-43fd-9abe-68c43a407ae2", 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6808", 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6809", 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6810", 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6811", 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6814", 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6815", 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6812", 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6813", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6809/1592419018", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6811/1592419018", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1592419018", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1592419018", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "373f8002-53a4-4ca7-bbbf-70e8d56b3571", 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 19, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6800", 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6801", 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6802", 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6803", 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6806", 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6807", 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6804", 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6805", 2024-09-16T22:16:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6801/3448871901", 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6803/3448871901", 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3448871901", 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3448871901", 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.033837+0000", 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.799099+0000", 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.769970+0000", 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.965029+0000", 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.706036+0000", 2024-09-16T22:16:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.158188+0000", 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.722931+0000", 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.485009+0000", 2024-09-16T22:16:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-16T22:16:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [ 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-16T22:16:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-16T22:16:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-16T22:16:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-16T22:16:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:14.217 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:14.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:14.220 INFO:tasks.thrashosds.thrasher:j is {'epoch': 23, 'fsid': '57fe7da5-e356-48a4-ba33-5558b1026a32', 'created': '2024-09-16T22:15:44.319460+0000', 'modified': '2024-09-16T22:16:14.110173+0000', 'last_up_change': '2024-09-16T22:15:56.374983+0000', 'last_in_change': '2024-09-16T22:15:46.438697+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T22:15:56.583719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T22:16:02.234097+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T22:16:13.789572+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '44de8de0-e16a-4325-94b9-1cc61f4829db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2585124124}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2585124124}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2585124124}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2585124124}]}, 'public_addr': '172.21.15.31:6817/2585124124', 'cluster_addr': '172.21.15.31:6819/2585124124', 'heartbeat_back_addr': '172.21.15.31:6823/2585124124', 'heartbeat_front_addr': '172.21.15.31:6821/2585124124', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3271285223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3271285223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3271285223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3271285223}]}, 'public_addr': '172.21.15.31:6801/3271285223', 'cluster_addr': '172.21.15.31:6803/3271285223', 'heartbeat_back_addr': '172.21.15.31:6807/3271285223', 'heartbeat_front_addr': '172.21.15.31:6805/3271285223', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b080bc29-0a92-4200-ae3f-0a17a1722145', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2874892035}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2874892035}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2874892035}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2874892035}]}, 'public_addr': '172.21.15.31:6809/2874892035', 'cluster_addr': '172.21.15.31:6811/2874892035', 'heartbeat_back_addr': '172.21.15.31:6815/2874892035', 'heartbeat_front_addr': '172.21.15.31:6813/2874892035', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0bc64c75-486c-40ac-abaf-381e151623bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3768411773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3768411773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3768411773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3768411773}]}, 'public_addr': '172.21.15.31:6825/3768411773', 'cluster_addr': '172.21.15.31:6827/3768411773', 'heartbeat_back_addr': '172.21.15.31:6831/3768411773', 'heartbeat_front_addr': '172.21.15.31:6829/3768411773', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 3421901083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3421901083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3421901083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3421901083}]}, 'public_addr': '172.21.15.92:6825/3421901083', 'cluster_addr': '172.21.15.92:6827/3421901083', 'heartbeat_back_addr': '172.21.15.92:6831/3421901083', 'heartbeat_front_addr': '172.21.15.92:6829/3421901083', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 335833224}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 335833224}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 335833224}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 335833224}]}, 'public_addr': '172.21.15.92:6817/335833224', 'cluster_addr': '172.21.15.92:6819/335833224', 'heartbeat_back_addr': '172.21.15.92:6823/335833224', 'heartbeat_front_addr': '172.21.15.92:6821/335833224', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '92f145b7-e868-43fd-9abe-68c43a407ae2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1592419018}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1592419018}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1592419018}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1592419018}]}, 'public_addr': '172.21.15.92:6809/1592419018', 'cluster_addr': '172.21.15.92:6811/1592419018', 'heartbeat_back_addr': '172.21.15.92:6815/1592419018', 'heartbeat_front_addr': '172.21.15.92:6813/1592419018', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '373f8002-53a4-4ca7-bbbf-70e8d56b3571', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3448871901}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3448871901}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3448871901}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3448871901}]}, 'public_addr': '172.21.15.92:6801/3448871901', 'cluster_addr': '172.21.15.92:6803/3448871901', 'heartbeat_back_addr': '172.21.15.92:6807/3448871901', 'heartbeat_front_addr': '172.21.15.92:6805/3448871901', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.033837+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.799099+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.769970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.965029+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.706036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.158188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.722931+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.485009+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T22:16:14.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:14.221 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:14.219+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:14.318 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:14.320+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:14.370 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.371 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:14.421 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:14.423+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:14.520 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:14.522+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:14.527 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.527 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:14.564 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:14.575 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T22:16:14.113366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":2965504,"data_stored":1776918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.545566"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430191+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:05.430191+0000","last_peered":"2024-09-16T22:16:05.430191+0000","last_clean":"2024-09-16T22:16:05.430191+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:05.430191+0000","last_undegraded":"2024-09-16T22:16:05.430191+0000","last_fullsized":"2024-09-16T22:16:05.430191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.824266+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:05.824266+0000","last_peered":"2024-09-16T22:16:05.824266+0000","last_clean":"2024-09-16T22:16:05.824266+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:05.824266+0000","last_undegraded":"2024-09-16T22:16:05.824266+0000","last_fullsized":"2024-09-16T22:16:05.824266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430483+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:05.430483+0000","last_peered":"2024-09-16T22:16:05.430483+0000","last_clean":"2024-09-16T22:16:05.430483+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:05.430483+0000","last_undegraded":"2024-09-16T22:16:05.430483+0000","last_fullsized":"2024-09-16T22:16:05.430483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426817+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:05.426817+0000","last_peered":"2024-09-16T22:16:05.426817+0000","last_clean":"2024-09-16T22:16:05.426817+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:05.426817+0000","last_undegraded":"2024-09-16T22:16:05.426817+0000","last_fullsized":"2024-09-16T22:16:05.426817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430960+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:05.430960+0000","last_peered":"2024-09-16T22:16:05.430960+0000","last_clean":"2024-09-16T22:16:05.430960+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:05.430960+0000","last_undegraded":"2024-09-16T22:16:05.430960+0000","last_fullsized":"2024-09-16T22:16:05.430960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.930317+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:05.930317+0000","last_peered":"2024-09-16T22:16:05.930317+0000","last_clean":"2024-09-16T22:16:05.930317+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:05.930317+0000","last_undegraded":"2024-09-16T22:16:05.930317+0000","last_fullsized":"2024-09-16T22:16:05.930317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.429704+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:05.429704+0000","last_peered":"2024-09-16T22:16:05.429704+0000","last_clean":"2024-09-16T22:16:05.429704+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:05.429704+0000","last_undegraded":"2024-09-16T22:16:05.429704+0000","last_fullsized":"2024-09-16T22:16:05.429704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.430542+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:05.430542+0000","last_peered":"2024-09-16T22:16:05.430542+0000","last_clean":"2024-09-16T22:16:05.430542+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:05.430542+0000","last_undegraded":"2024-09-16T22:16:05.430542+0000","last_fullsized":"2024-09-16T22:16:05.430542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T22:16:05.426110+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:05.426110+0000","last_peered":"2024-09-16T22:16:05.426110+0000","last_clean":"2024-09-16T22:16:05.426110+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:05.426110+0000","last_undegraded":"2024-09-16T22:16:05.426110+0000","last_fullsized":"2024-09-16T22:16:05.426110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T22:16:14.110173+0000","last_change":"2024-09-16T22:16:14.110173+0000","last_active":"2024-09-16T22:16:14.110173+0000","last_peered":"2024-09-16T22:16:14.110173+0000","last_clean":"2024-09-16T22:16:14.110173+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T22:16:14.110173+0000","last_undegraded":"2024-09-16T22:16:14.110173+0000","last_fullsized":"2024-09-16T22:16:14.110173+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:14.578 INFO:tasks.thrashosds.thrasher:Setting 3.5 to [4, 2, 0, 5] 2024-09-16T22:16:14.578 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.5', '4', '2', '0', '5'] 2024-09-16T22:16:14.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.5 4 2 0 5 2024-09-16T22:16:14.621 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:14.623+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:14.646 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.647 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:14.722 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:14.724+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:14.806 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.806 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:14.861 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:14.863+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:14.924 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:14.925+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:14.925 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:14.925 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:14.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:15.024 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:15.026+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:15.082 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.082 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.5 pg_upmap_items mapping to [4->2,0->5] 2024-09-16T22:16:15.145 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:15.146+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:15.226 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:15.228+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:15.232 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.232 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:15.327 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:15.328+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:15.428 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:15.429+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:15.529 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:15.531+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:15.544 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.545 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:15.630 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:15.632+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:15.674 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.674 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:15.768 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:15.770+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:15.831 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.831 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.832 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:15.833+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:15.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:15.933 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:15.934+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:15.981 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:15.981 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:15.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:16.034 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:16.036+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:16.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:16.092 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:16.092 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:16.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:16.135 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:16.136+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:16.188 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:16.189 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:16.205 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:16.205 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:16.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:16:16.221 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:16.222 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:16.236 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:16.237+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:16.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:16.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:16.244 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:16.244 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:16.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:16.257 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:16.257 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:16.274 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:16.291 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:16.308 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:16.337 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:16.339+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2024-09-16T22:16:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.402451+0000", 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.994619264000001, 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042680249999999999, 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402451+0000", 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402451+0000", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402451+0000", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.28e-07 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402457+0000", 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402465+0000", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1769999999999998e-06 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402475+0000", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5920000000000007e-06 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402528+0000", 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2772000000000002e-05 2024-09-16T22:16:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.402556+0000", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8725999999999999e-05 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.406680+0000", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041233989999999998 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.406719+0000", 2024-09-16T22:16:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9017999999999998e-05 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:42 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'24, mlcod=17'19)", 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.864981766, 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056317980000000004, 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532089+0000", 2024-09-16T22:16:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3300000000000001e-07 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532096+0000", 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0450000000000003e-06 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532106+0000", 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532111+0000", 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2950000000000001e-06 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532156+0000", 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4672000000000002e-05 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532171+0000", 2024-09-16T22:16:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5392000000000001e-05 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537693+0000", 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055218339999999998 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537720+0000", 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7285e-05 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:43 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'25, mlcod=17'21)", 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.532810+0000", 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.864260277, 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049440899999999999, 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532810+0000", 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532810+0000", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532810+0000", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2399999999999998e-07 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532814+0000", 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8689999999999997e-06 2024-09-16T22:16:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532821+0000", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0879999999999997e-06 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532825+0000", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5149999999999998e-06 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532868+0000", 2024-09-16T22:16:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3136000000000002e-05 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532881+0000", 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3467e-05 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537723+0000", 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048419350000000003 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537754+0000", 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0555999999999997e-05 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:44 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'26, mlcod=17'21)", 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.533248+0000", 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.863821782999999, 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045440749999999999, 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533248+0000", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533248+0000", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533249+0000", 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-09-16T22:16:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533307+0000", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8411e-05 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533315+0000", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533320+0000", 2024-09-16T22:16:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6990000000000002e-06 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533358+0000", 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8346999999999997e-05 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533372+0000", 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3648e-05 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537756+0000", 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043846120000000004 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537792+0000", 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5951000000000002e-05 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:45 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'27, mlcod=17'21)", 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.533781+0000", 2024-09-16T22:16:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.863288516000001, 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047443269999999996, 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533781+0000", 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533781+0000", 2024-09-16T22:16:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533782+0000", 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533785+0000", 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.909e-06 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533791+0000", 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.533794+0000", 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3569999999999998e-06 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534204+0000", 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000410107 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534218+0000", 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3961999999999999e-05 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538499+0000", 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004280987 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538526+0000", 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6795e-05 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:46 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'28, mlcod=17'21)", 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.534335+0000", 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.862734721999999, 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042235049999999998, 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534335+0000", 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534335+0000", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534336+0000", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534342+0000", 2024-09-16T22:16:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.874e-06 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534356+0000", 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4612e-05 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534363+0000", 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5729999999999996e-06 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534495+0000", 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013175400000000001 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534510+0000", 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4825e-05 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538529+0000", 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040189040000000002 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538559+0000", 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:47 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'29, mlcod=17'21)", 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.534775+0000", 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.862294652999999, 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003813122, 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534775+0000", 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534775+0000", 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534776+0000", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.54e-07 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534859+0000", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3218000000000002e-05 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534870+0000", 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0485e-05 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534876+0000", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6039999999999998e-06 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534919+0000", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2592000000000001e-05 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.534932+0000", 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3112000000000001e-05 2024-09-16T22:16:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538561+0000", 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036293900000000001 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538588+0000", 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6967000000000001e-05 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'21)", 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.535304+0000", 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.861766439, 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052945259999999999, 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535304+0000", 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535304+0000", 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535304+0000", 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535309+0000", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999997e-06 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535319+0000", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535324+0000", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6990000000000002e-06 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537205+0000", 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018815139999999999 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537226+0000", 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0098999999999998e-05 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540571+0000", 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033451069999999999 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540598+0000", 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7429000000000001e-05 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.535822+0000", 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.861247782, 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048140120000000003, 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535822+0000", 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535822+0000", 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535823+0000", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.46e-07 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535846+0000", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3332e-05 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535854+0000", 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9799999999999998e-06 2024-09-16T22:16:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.535858+0000", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5210000000000004e-06 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537493+0000", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016348280000000001 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537506+0000", 2024-09-16T22:16:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2777e-05 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540600+0000", 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003094383 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540636+0000", 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5744999999999997e-05 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.536257+0000", 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.860813201999999, 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046860419999999996, 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536257+0000", 2024-09-16T22:16:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536257+0000", 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536257+0000", 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6499999999999999e-07 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536322+0000", 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4252999999999996e-05 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536331+0000", 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0070000000000003e-06 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536339+0000", 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0800000000000006e-06 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537838+0000", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014991200000000001 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537854+0000", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6129000000000001e-05 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540909+0000", 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003054649 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540943+0000", 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4239000000000002e-05 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.536807+0000", 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.860262949999999, 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041872150000000002, 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536807+0000", 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536807+0000", 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536808+0000", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536816+0000", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.729e-06 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536848+0000", 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2472999999999999e-05 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.536858+0000", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2960000000000004e-06 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538294+0000", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001436648 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538307+0000", 2024-09-16T22:16:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2897e-05 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540945+0000", 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002637749 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540994+0000", 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9345999999999998e-05 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'22)", 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.537220+0000", 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.859849718, 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037970550000000001, 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537220+0000", 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537220+0000", 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537221+0000", 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537227+0000", 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8279999999999999e-06 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537236+0000", 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6150000000000003e-06 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537243+0000", 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1229999999999997e-06 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538616+0000", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013732939999999999 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.538630+0000", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4324e-05 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.540997+0000", 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023665549999999998 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.541017+0000", 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-05 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:81 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'59, mlcod=17'57)", 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.863020+0000", 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.534049576000001, 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034672980000000002, 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863020+0000", 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863020+0000", 2024-09-16T22:16:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863021+0000", 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1600000000000001e-07 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863025+0000", 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1420000000000003e-06 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863034+0000", 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0620000000000007e-06 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863040+0000", 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863077+0000", 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6838000000000002e-05 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.863089+0000", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2472e-05 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.866458+0000", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033684100000000001 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.866488+0000", 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0120999999999998e-05 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:90 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=17'60)", 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.266967697999998, 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056948570000000002, 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130103+0000", 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130163+0000", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0516999999999997e-05 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130172+0000", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3990000000000004e-06 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130177+0000", 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8960000000000002e-06 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130210+0000", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3492999999999999e-05 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130220+0000", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0176999999999999e-05 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135746+0000", 2024-09-16T22:16:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055257730000000003 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135797+0000", 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0982000000000001e-05 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:91 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'65, mlcod=17'60)", 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.130620+0000", 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.266449953999999, 2024-09-16T22:16:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052164569999999999, 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130620+0000", 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130620+0000", 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130620+0000", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130624+0000", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5580000000000001e-06 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130629+0000", 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3619999999999997e-06 2024-09-16T22:16:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130632+0000", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1489999999999998e-06 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130657+0000", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130669+0000", 2024-09-16T22:16:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1911e-05 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135799+0000", 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051304109999999997 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135837+0000", 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7004e-05 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:92 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'61)", 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.131076+0000", 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.265994385999999, 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047885849999999997, 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131076+0000", 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131076+0000", 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131076+0000", 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131080+0000", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1010000000000002e-06 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131086+0000", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7559999999999996e-06 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131090+0000", 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0099999999999997e-06 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131123+0000", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2616000000000001e-05 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131134+0000", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1242e-05 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135838+0000", 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047038959999999999 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135864+0000", 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6542000000000001e-05 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:93 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'61)", 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.131445+0000", 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.265624663000001, 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044478390000000003, 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131445+0000", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131445+0000", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131446+0000", 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131501+0000", 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5782999999999998e-05 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131507+0000", 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7640000000000002e-06 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131511+0000", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.856e-06 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131540+0000", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8549999999999999e-05 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131550+0000", 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0890000000000001e-05 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135865+0000", 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043146749999999996 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135893+0000", 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8021000000000002e-05 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-16T22:16:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.131889+0000", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.265180775000001, 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040301, 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131889+0000", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131889+0000", 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131890+0000", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131897+0000", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131903+0000", 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9229999999999999e-06 2024-09-16T22:16:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131907+0000", 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7689999999999998e-06 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131937+0000", 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9818000000000001e-05 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.131949+0000", 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2129e-05 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135894+0000", 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039451929999999996 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135919+0000", 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5253e-05 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.137539+0000", 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.259530877, 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040703969999999999, 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137539+0000", 2024-09-16T22:16:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137539+0000", 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137540+0000", 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2500000000000005e-07 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137586+0000", 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6582999999999998e-05 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137596+0000", 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0101e-05 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137601+0000", 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3800000000000004e-06 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137634+0000", 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2845999999999997e-05 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.137646+0000", 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2223e-05 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.141560+0000", 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003914074 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.141610+0000", 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9564999999999997e-05 2024-09-16T22:16:16.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.142244+0000", 2024-09-16T22:16:16.483 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.254826134999998, 2024-09-16T22:16:16.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049770700000000001, 2024-09-16T22:16:16.484 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.484 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142244+0000", 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142244+0000", 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142244+0000", 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142249+0000", 2024-09-16T22:16:16.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:16:16.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142259+0000", 2024-09-16T22:16:16.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9289999999999993e-06 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142264+0000", 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6410000000000001e-06 2024-09-16T22:16:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142297+0000", 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3769999999999997e-05 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.142309+0000", 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.153e-05 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.147143+0000", 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048342979999999999 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.147221+0000", 2024-09-16T22:16:16.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7813999999999997e-05 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.493 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:16.495 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:16.496+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:16.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:16.596 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:16.598+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:16.640 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:16.640 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:16.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:16.697 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:16.698+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:16.791 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:16.792 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:16.797 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:16.799+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:16.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:16.898 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:16.899+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:16.942 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:16.942 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:16.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:16.942 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4217.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:04.419654+0000", 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.525101892, 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001016713, 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419654+0000", 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419654+0000", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419654+0000", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419661+0000", 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0620000000000003e-06 2024-09-16T22:16:16.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419679+0000", 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7890999999999999e-05 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419690+0000", 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1236999999999999e-05 2024-09-16T22:16:16.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419769+0000", 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8176000000000005e-05 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.419799+0000", 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0957000000000002e-05 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.420626+0000", 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082664800000000003 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:04.420671+0000", 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4410999999999999e-05 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4217.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:05.429781+0000", 2024-09-16T22:16:16.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.51497468, 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097953100000000006, 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429781+0000", 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429781+0000", 2024-09-16T22:16:16.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429781+0000", 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-07 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429787+0000", 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429805+0000", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8090000000000001e-05 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429815+0000", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0548e-05 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429919+0000", 2024-09-16T22:16:16.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010356 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.429945+0000", 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6267000000000001e-05 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.430720+0000", 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00077477999999999998 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:16.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:05.430761+0000", 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0478000000000001e-05 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:16.952 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:16.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:16.999 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:17.000+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:17.055 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:17.056 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:17.056 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:17.056 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:17.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:17.105 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:17.107+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:17.125 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:16:17.200 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:17.202+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:17.209 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:17.210 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:17.210 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:17.210 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:17.210 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:17.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:17.303 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:17.305+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:17.362 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:17.362 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2024-09-16T22:16:17.362 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:17.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:17.411 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:17.413+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:17.478 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:17.478 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:17.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:17.502 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:17.504+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:17.594 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:17.594 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:17.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:17.603 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:17.604+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:17.704 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:17.705+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.57948758099999997, 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075317699999999995, 2024-09-16T22:16:17.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133779+0000", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3900000000000001e-07 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133782+0000", 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6689999999999999e-06 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133793+0000", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0499000000000001e-05 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133801+0000", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7989999999999994e-06 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133864+0000", 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799000000000004e-05 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133887+0000", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2289e-05 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.134492+0000", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060517100000000005 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.134532+0000", 2024-09-16T22:16:17.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712000000000002e-05 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:17.716 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:17.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:17.804 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:17.806+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:17.866 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:17.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:17.867 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:17.867 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:17.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:17.906 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:17.907+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:18.007 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:18.008+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:18.022 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:18.023 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:18.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:18.115 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:18.117+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:18.181 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.790727500999999, 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015694280000000001, 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391488+0000", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6279999999999999e-06 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391496+0000", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9119999999999998e-06 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391507+0000", 2024-09-16T22:16:18.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0251000000000001e-05 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391515+0000", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1890000000000007e-06 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391562+0000", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7051000000000001e-05 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391570+0000", 2024-09-16T22:16:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1899999999999995e-06 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.401974+0000", 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010404096999999999 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402150+0000", 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017572 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402318+0000", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016847499999999999 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402960+0000", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064145900000000004 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407117+0000", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041574799999999999 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407153+0000", 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5587000000000003e-05 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407181+0000", 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8240999999999999e-05 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.652087381000001, 2024-09-16T22:16:18.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077487449999999996, 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:18.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530130+0000", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530135+0000", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6659999999999996e-06 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530146+0000", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0379e-05 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531751+0000", 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001605022 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531840+0000", 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9159000000000005e-05 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531986+0000", 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000145978 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534511+0000", 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025251330000000002 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537823+0000", 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033120950000000001 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537855+0000", 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1955999999999997e-05 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537876+0000", 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0789999999999999e-05 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651922461000002, 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007677835, 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 43 2024-09-16T22:16:18.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:18.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530299+0000", 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530304+0000", 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530307+0000", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.095e-06 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532608+0000", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023004319999999998 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532653+0000", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5506000000000001e-05 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532734+0000", 2024-09-16T22:16:18.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0937000000000001e-05 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534517+0000", 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001782596 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537905+0000", 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033881570000000001 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537945+0000", 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0624000000000003e-05 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537970+0000", 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4216999999999999e-05 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:18.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651843803999999, 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076823799999999999, 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 44 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530444+0000", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2640999999999998e-05 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530451+0000", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530455+0000", 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0690000000000003e-06 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533032+0000", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025771930000000002 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533083+0000", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1218999999999999e-05 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533162+0000", 2024-09-16T22:16:18.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8898000000000003e-05 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535469+0000", 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023075109999999999 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538005+0000", 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002535388 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538032+0000", 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6954e-05 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538053+0000", 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1080000000000001e-05 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:18.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651752942000002, 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081949469999999993, 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530462+0000", 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530464+0000", 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530468+0000", 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5410000000000001e-06 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530471+0000", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533574+0000", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031031169999999999 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533631+0000", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533707+0000", 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6099000000000004e-05 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535480+0000", 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017729169999999999 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538616+0000", 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031358829999999999 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538639+0000", 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3315e-05 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538656+0000", 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7084000000000001e-05 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651738994999999, 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082594290000000004, 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 46 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530476+0000", 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6899999999999999e-07 2024-09-16T22:16:18.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530478+0000", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530481+0000", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3689999999999998e-06 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530483+0000", 2024-09-16T22:16:18.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534007+0000", 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035237409999999999 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534089+0000", 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534239+0000", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149615 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536005+0000", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017664099999999999 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538687+0000", 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026822949999999999 2024-09-16T22:16:18.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538714+0000", 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7236000000000001e-05 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538735+0000", 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0293e-05 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651700452, 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082918049999999993, 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 47 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530608+0000", 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4163e-05 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530614+0000", 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530617+0000", 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534562+0000", 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039451900000000003 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534617+0000", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5281999999999997e-05 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534695+0000", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7344999999999999e-05 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536013+0000", 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013184970000000001 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538757+0000", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027440049999999999 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538787+0000", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9734e-05 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538806+0000", 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8678e-05 2024-09-16T22:16:18.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651590890000001, 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010151530000000001, 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 48 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530624+0000", 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4400000000000001e-07 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530626+0000", 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530629+0000", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3519999999999998e-06 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530632+0000", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535102+0000", 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044695769999999998 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535147+0000", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5874999999999997e-05 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535225+0000", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7262000000000007e-05 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536945+0000", 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001720297 2024-09-16T22:16:18.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540728+0000", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003783472 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540754+0000", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5791000000000001e-05 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540775+0000", 2024-09-16T22:16:18.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0733999999999998e-05 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651558001000001, 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010193045, 2024-09-16T22:16:18.231 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:18.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530657+0000", 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530660+0000", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4180000000000001e-06 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530666+0000", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7370000000000003e-06 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530674+0000", 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0399999999999993e-06 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535615+0000", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049413870000000002 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535666+0000", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.092e-05 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535755+0000", 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8263999999999994e-05 2024-09-16T22:16:18.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536950+0000", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001195776 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540802+0000", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038521749999999998 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540828+0000", 2024-09-16T22:16:18.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5731000000000001e-05 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540849+0000", 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1182000000000001e-05 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:18.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651475656999999, 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010330358, 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:18.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530828+0000", 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8504999999999994e-05 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530835+0000", 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7179999999999996e-06 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530839+0000", 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536044+0000", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052045609999999999 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536092+0000", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100999999999999e-05 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536171+0000", 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9357999999999995e-05 2024-09-16T22:16:18.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536954+0000", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078211999999999997 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541028+0000", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040741529999999996 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541052+0000", 2024-09-16T22:16:18.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4267e-05 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541069+0000", 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7088999999999999e-05 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651367610000001, 2024-09-16T22:16:18.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010291738999999999, 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.39e-07 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530849+0000", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2639999999999998e-06 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530853+0000", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5279999999999999e-06 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530856+0000", 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-16T22:16:18.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536566+0000", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057098060000000004 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536617+0000", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0711000000000003e-05 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536691+0000", 2024-09-16T22:16:18.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4536999999999995e-05 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537339+0000", 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064814099999999997 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541092+0000", 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037528790000000002 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541117+0000", 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5279999999999999e-05 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541138+0000", 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1047999999999999e-05 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.651354194, 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010348868000000001, 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 52 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530863+0000", 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530867+0000", 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530879+0000", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536976+0000", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060963989999999997 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537037+0000", 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1623000000000003e-05 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537121+0000", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3033000000000003e-05 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537655+0000", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053447399999999995 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541165+0000", 2024-09-16T22:16:18.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035098120000000002 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541190+0000", 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5313e-05 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541209+0000", 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8878000000000001e-05 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337335319000001, 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089954470000000002, 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 70 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:18.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0199999999999998e-07 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844882+0000", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844886+0000", 2024-09-16T22:16:18.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844889+0000", 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848458+0000", 2024-09-16T22:16:18.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035685589999999998 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848544+0000", 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6298000000000006e-05 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848675+0000", 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130808 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851222+0000", 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025471500000000002 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853828+0000", 2024-09-16T22:16:18.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026053309999999998 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853852+0000", 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.495e-05 2024-09-16T22:16:18.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853874+0000", 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1982999999999999e-05 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337320937000001, 2024-09-16T22:16:18.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090279209999999995, 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 71 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:18.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844894+0000", 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844896+0000", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.069e-06 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844899+0000", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844902+0000", 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-16T22:16:18.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849147+0000", 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042455749999999997 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849225+0000", 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7844999999999998e-05 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849353+0000", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012841400000000001 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851381+0000", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020275390000000001 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853879+0000", 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024981949999999999 2024-09-16T22:16:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853900+0000", 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1138e-05 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853921+0000", 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1029000000000001e-05 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337250391000001, 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011701283, 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 72 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-07 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845021+0000", 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6480000000000001e-05 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845026+0000", 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3449999999999996e-06 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845033+0000", 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:18.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849957+0000", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049242820000000003 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850000+0000", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3381999999999999e-05 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850076+0000", 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-09-16T22:16:18.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853789+0000", 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00371274 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856571+0000", 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002782432 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856623+0000", 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1907000000000003e-05 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856665+0000", 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2347e-05 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337175062, 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011700838, 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 73 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845040+0000", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845042+0000", 2024-09-16T22:16:18.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845047+0000", 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0860000000000001e-06 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845057+0000", 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7149999999999993e-06 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851416+0000", 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063595580000000004 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851460+0000", 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510999999999997e-05 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851531+0000", 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1266000000000005e-05 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853820+0000", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022893879999999998 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856673+0000", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028527359999999998 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856720+0000", 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7027000000000002e-05 2024-09-16T22:16:18.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856740+0000", 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9973999999999999e-05 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337149477000001, 2024-09-16T22:16:18.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011717971000000001, 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 74 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:18.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845070+0000", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000004e-06 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845074+0000", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9519999999999996e-06 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845076+0000", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000001e-06 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851800+0000", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067234740000000001 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851883+0000", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2935999999999993e-05 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852017+0000", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013409199999999999 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853945+0000", 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019281179999999999 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856745+0000", 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002800276 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856766+0000", 2024-09-16T22:16:18.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0370999999999999e-05 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856783+0000", 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7132000000000001e-05 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337133414, 2024-09-16T22:16:18.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014248581999999999, 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 75 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:18.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845140+0000", 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9061000000000003e-05 2024-09-16T22:16:18.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845146+0000", 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845150+0000", 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5899999999999999e-06 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852357+0000", 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072068239999999997 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852403+0000", 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5924999999999998e-05 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852502+0000", 2024-09-16T22:16:18.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9129000000000006e-05 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854543+0000", 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020415730000000001 2024-09-16T22:16:18.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859274+0000", 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047301799999999996 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859305+0000", 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1117e-05 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859329+0000", 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4760999999999999e-05 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.337055576000001, 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014219621, 2024-09-16T22:16:18.301 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 76 2024-09-16T22:16:18.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:16:18.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845165+0000", 2024-09-16T22:16:18.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999998e-06 2024-09-16T22:16:18.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845169+0000", 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9589999999999997e-06 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845172+0000", 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.709e-06 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853983+0000", 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088106799999999996 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854065+0000", 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2452000000000003e-05 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854200+0000", 2024-09-16T22:16:18.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013444600000000001 2024-09-16T22:16:18.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.855726+0000", 2024-09-16T22:16:18.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015263049999999999 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859334+0000", 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036082890000000002 2024-09-16T22:16:18.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859355+0000", 2024-09-16T22:16:18.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0820999999999999e-05 2024-09-16T22:16:18.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859378+0000", 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3068e-05 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:18.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:18.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.052835311999999, 2024-09-16T22:16:18.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071099739999999998, 2024-09-16T22:16:18.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:18.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:18.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:18.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 94 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:18.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:18.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129380+0000", 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129383+0000", 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8850000000000001e-06 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129390+0000", 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129396+0000", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131718+0000", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023225619999999998 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131759+0000", 2024-09-16T22:16:18.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0601e-05 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131829+0000", 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9830000000000001e-05 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.135241+0000", 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003412567 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136401+0000", 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011601630000000001 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136452+0000", 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1113999999999999e-05 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136489+0000", 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6575999999999997e-05 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:18.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:18.324 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:18.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:18.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:18.326 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:18.326+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:18.426 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:18.427+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:18.480 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:18.480 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:18.480 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:18.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:18.501 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:18.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:18.526 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:18.528+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:18.537 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:18.537 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:18.554 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:18.554 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:18.571 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:18.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:18.601 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:18.602 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:18.606 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:18.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:18.623 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:18.626 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:18.628+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:18.727 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:18.729+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:18.731 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:18.731 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:18.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:18.828 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:18.829+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:18.856 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:18.856 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:18.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:18.870 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0x631c1d9 2024-09-16T22:16:18.877 INFO:teuthology.orchestra.run.smithi031.stdout:1 trunc from 0x0 to 0x631c1d9 2024-09-16T22:16:18.885 INFO:teuthology.orchestra.run.smithi031.stdout:5 writesame 0x494061e thru 0x494b2af data_size 0x25(0xac92 bytes) 2024-09-16T22:16:18.929 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:18.931+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:18.976 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:18.976 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:18.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:19.068 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:19.070+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:19.094 INFO:teuthology.orchestra.run.smithi031.stdout:6 read 0x5f86ebf thru 0x5f93bef (0xcd31 bytes) 2024-09-16T22:16:19.094 INFO:teuthology.orchestra.run.smithi031.stdout:7 writesame 0x462a422 thru 0x46305e6 data_size 0x1b(0x61c5 bytes) 2024-09-16T22:16:19.100 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:19.108 INFO:teuthology.orchestra.run.smithi031.stdout:9 writesame 0x646e8a4 thru 0x647051e data_size 0x1(0x1c7b bytes) 2024-09-16T22:16:19.128 INFO:teuthology.orchestra.run.smithi031.stdout:10 write 0xeb239e7 thru 0xeb2f9ff (0xc019 bytes) 2024-09-16T22:16:19.131 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:19.133+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:19.133 INFO:teuthology.orchestra.run.smithi031.stdout:11 writesame 0x56a6810 thru 0x56a9ad2 data_size 0x17(0x32c3 bytes) 2024-09-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi031.stdout:12 punch from 0xe7512a9 to 0xe753e0a, (0x2b61 bytes) 2024-09-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi031.stdout:14 read 0x490ee8e thru 0x491acea (0xbe5d bytes) 2024-09-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi031.stdout:15 punch from 0x46f1e44 to 0x46f27f6, (0x9b2 bytes) 2024-09-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi031.stdout:16 write 0x417be4f thru 0x4187733 (0xb8e5 bytes) 2024-09-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.142 INFO:teuthology.orchestra.run.smithi031.stdout:17 write 0x361481e thru 0x361fa4c (0xb22f bytes) 2024-09-16T22:16:19.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:16:19.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:19.148 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe705d10 2024-09-16T22:16:19.148 INFO:teuthology.orchestra.run.smithi031.stdout:20 trunc from 0xeb2fa00 to 0xe705d10 2024-09-16T22:16:19.160 INFO:teuthology.orchestra.run.smithi031.stdout:25 clone 1 order 18 su 32768 sc 6 2024-09-16T22:16:19.232 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:19.234+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:19.261 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.261 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:19.354 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:19.356+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:19.413 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.413 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:19.462 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:19.464+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:19.534 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:19.536+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:19.564 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.564 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:19.656 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:19.658+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:19.715 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.715 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:19.735 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:19.737+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:19.836 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:19.838+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:19.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:19.959 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:19.961+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:20.019 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.019 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:20.038 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:20.040+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:20.136 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:16:20.139 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:20.141+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:20.145 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:20.145 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:20.146 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T22:16:20.146 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:16:20.146 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:16:20.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T22:16:20.176 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.176 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:20.266 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:20.266 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:16:20.266 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:20.270 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:20.272+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:20.330 INFO:teuthology.orchestra.run.smithi031.stdout:26 read 0x3144bcc thru 0x3147d38 (0x316d bytes) 2024-09-16T22:16:20.331 INFO:teuthology.orchestra.run.smithi031.stdout:27 read 0xb7ed817 thru 0xb7fc0af (0xe899 bytes) 2024-09-16T22:16:20.331 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.332 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.332 INFO:teuthology.orchestra.run.smithi031.stdout:28 read 0xe5211ef thru 0xe5235c6 (0x23d8 bytes) 2024-09-16T22:16:20.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:20.341 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:20.342+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:20.346 INFO:teuthology.orchestra.run.smithi031.stdout:30 trunc from 0xe705d10 to 0x1eaebdd 2024-09-16T22:16:20.367 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.367 INFO:teuthology.orchestra.run.smithi031.stdout:32 punch from 0x4a6328 to 0x4b582e, (0xf506 bytes) 2024-09-16T22:16:20.367 INFO:teuthology.orchestra.run.smithi031.stdout:33 trunc from 0x1eaebdd to 0x20abfec 2024-09-16T22:16:20.373 INFO:teuthology.orchestra.run.smithi031.stdout:34 punch from 0x19f574c to 0x19fac1a, (0x54ce bytes) 2024-09-16T22:16:20.373 INFO:teuthology.orchestra.run.smithi031.stdout:35 punch from 0x2db233 to 0x2e38ae, (0x867b bytes) 2024-09-16T22:16:20.373 INFO:teuthology.orchestra.run.smithi031.stdout:36 write 0x6d340f thru 0x6d3652 (0x244 bytes) 2024-09-16T22:16:20.378 INFO:teuthology.orchestra.run.smithi031.stdout:37 writesame 0x84bc9a thru 0x8586d9 data_size 0x20(0xca40 bytes) 2024-09-16T22:16:20.383 INFO:teuthology.orchestra.run.smithi031.stdout:38 read 0x1cad6da thru 0x1cb3786 (0x60ad bytes) 2024-09-16T22:16:20.383 INFO:teuthology.orchestra.run.smithi031.stdout:39 write 0xc89ec2 thru 0xc966e2 (0xc821 bytes) 2024-09-16T22:16:20.388 INFO:teuthology.orchestra.run.smithi031.stdout:40 read 0xb7bddc thru 0xb8bbe3 (0xfe08 bytes) 2024-09-16T22:16:20.399 INFO:teuthology.orchestra.run.smithi031.stdout:41 trunc from 0x20abfec to 0xc1e8ae7 2024-09-16T22:16:20.406 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.417 INFO:teuthology.orchestra.run.smithi031.stdout:43 trunc from 0xc1e8ae7 to 0x28d83b1 2024-09-16T22:16:20.441 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:20.443+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:44 write 0xdb403c7 thru 0xdb4f0e9 (0xed23 bytes) 2024-09-16T22:16:20.447 INFO:teuthology.orchestra.run.smithi031.stdout:45 read 0xd9ce18a thru 0xd9d4fe6 (0x6e5d bytes) 2024-09-16T22:16:20.447 INFO:teuthology.orchestra.run.smithi031.stdout:49 read 0x4fd8edb thru 0x4fdd39b (0x44c1 bytes) 2024-09-16T22:16:20.452 INFO:teuthology.orchestra.run.smithi031.stdout:50 trunc from 0xdb4f0ea to 0x91f65c1 2024-09-16T22:16:20.462 INFO:teuthology.orchestra.run.smithi031.stdout:52 read 0x611caac thru 0x61207f8 (0x3d4d bytes) 2024-09-16T22:16:20.472 INFO:teuthology.orchestra.run.smithi031.stdout:53 trunc from 0x91f65c1 to 0xcd803e 2024-09-16T22:16:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.492 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:20.521 INFO:teuthology.orchestra.run.smithi031.stdout:54 write 0x83c4a9a thru 0x83d2ae3 (0xe04a bytes) 2024-09-16T22:16:20.529 INFO:teuthology.orchestra.run.smithi031.stdout:55 write 0x309a7aa thru 0x30a585d (0xb0b4 bytes) 2024-09-16T22:16:20.533 INFO:teuthology.orchestra.run.smithi031.stdout:57 write 0x5d58b4d thru 0x5d5c40c (0x38c0 bytes) 2024-09-16T22:16:20.536 INFO:teuthology.orchestra.run.smithi031.stdout:58 write 0xae7a9c thru 0xaf1ec2 (0xa427 bytes) 2024-09-16T22:16:20.546 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:20.548+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:59 trunc from 0x83d2ae4 to 0x423e723 2024-09-16T22:16:20.571 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.584 INFO:teuthology.orchestra.run.smithi031.stdout:63 writesame 0x770ddd7 thru 0x7714935 data_size 0x1(0x6b5f bytes) 2024-09-16T22:16:20.584 INFO:teuthology.orchestra.run.smithi031.stdout:65 writesame 0x5a8adc9 thru 0x5a90577 data_size 0x1(0x57af bytes) 2024-09-16T22:16:20.610 INFO:teuthology.orchestra.run.smithi031.stdout:66 write 0x8099f31 thru 0x809ea80 (0x4b50 bytes) 2024-09-16T22:16:20.613 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.613 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.616 INFO:teuthology.orchestra.run.smithi031.stdout:67 write 0x39e4f4f thru 0x39ea8e4 (0x5996 bytes) 2024-09-16T22:16:20.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:20.624 INFO:teuthology.orchestra.run.smithi031.stdout:68 writesame 0x66c496f thru 0x66c832e data_size 0x8(0x39c0 bytes) 2024-09-16T22:16:20.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:20.639 INFO:teuthology.orchestra.run.smithi031.stdout:71 writesame 0x8d9cc27 thru 0x8dabc5b data_size 0x1(0xf035 bytes) 2024-09-16T22:16:20.642 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:20.644+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:72 write 0xb3eb674 thru 0xb3f6734 (0xb0c1 bytes) 2024-09-16T22:16:20.652 INFO:teuthology.orchestra.run.smithi031.stdout:73 read 0x60e2626 thru 0x60eceda (0xa8b5 bytes) 2024-09-16T22:16:20.662 INFO:teuthology.orchestra.run.smithi031.stdout:74 trunc from 0xb3f6735 to 0x2d99319 2024-09-16T22:16:20.676 INFO:teuthology.orchestra.run.smithi031.stdout:75 punch from 0x11dd704 to 0x11ebee3, (0xe7df bytes) 2024-09-16T22:16:20.679 INFO:teuthology.orchestra.run.smithi031.stdout:77 trunc from 0x2d99319 to 0xa95dad 2024-09-16T22:16:20.714 INFO:teuthology.orchestra.run.smithi031.stdout:78 write 0xb0c7123 thru 0xb0cd772 (0x6650 bytes) 2024-09-16T22:16:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:81 punch from 0x71c3076 to 0x71c8ac6, (0x5a50 bytes) 2024-09-16T22:16:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:82 read 0x84aa9fe thru 0x84b5c99 (0xb29c bytes) 2024-09-16T22:16:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:83 write 0x664653a thru 0x6652f61 (0xca28 bytes) 2024-09-16T22:16:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:84 read 0x2426420 thru 0x2428209 (0x1dea bytes) 2024-09-16T22:16:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:85 write 0xa213650 thru 0xa222760 (0xf111 bytes) 2024-09-16T22:16:20.740 INFO:teuthology.orchestra.run.smithi031.stdout:86 trunc from 0xb0cd773 to 0x2f60e3d 2024-09-16T22:16:20.743 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:20.744+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:20.753 INFO:teuthology.orchestra.run.smithi031.stdout:87 writesame 0x1955ff thru 0x1a3bf1 data_size 0x25(0xe5f3 bytes) 2024-09-16T22:16:20.771 INFO:teuthology.orchestra.run.smithi031.stdout:88 trunc from 0x2f60e3d to 0xcb5fe68 2024-09-16T22:16:20.776 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:20.777 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.777 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.778 INFO:teuthology.orchestra.run.smithi031.stdout:89 trunc from 0xcb5fe68 to 0xdcc503f 2024-09-16T22:16:20.782 INFO:teuthology.orchestra.run.smithi031.stdout:90 flatten 2024-09-16T22:16:20.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:20.788 INFO:teuthology.orchestra.run.smithi031.stdout:92 write 0x126170 thru 0x1319fa (0xb88b bytes) 2024-09-16T22:16:20.795 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.795 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:20.795 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:20.796 INFO:teuthology.orchestra.run.smithi031.stdout:94 write 0x45d7f50 thru 0x45e3e58 (0xbf09 bytes) 2024-09-16T22:16:20.807 INFO:teuthology.orchestra.run.smithi031.stdout:95 read 0x831de60 thru 0x8322330 (0x44d1 bytes) 2024-09-16T22:16:20.807 INFO:teuthology.orchestra.run.smithi031.stdout:96 read 0x22f6f0a thru 0x22ff829 (0x8920 bytes) 2024-09-16T22:16:20.807 INFO:teuthology.orchestra.run.smithi031.stdout:97 read 0x44e9a40 thru 0x44ea623 (0xbe4 bytes) 2024-09-16T22:16:20.808 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:20.808 INFO:teuthology.orchestra.run.smithi031.stdout:99 read 0xc54e2f4 thru 0xc54e381 (0x8e bytes) 2024-09-16T22:16:20.808 INFO:teuthology.orchestra.run.smithi031.stdout:100 write 0x7a983cc thru 0x7aa40a9 (0xbcde bytes) 2024-09-16T22:16:20.811 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:20.817 INFO:teuthology.orchestra.run.smithi031.stdout:101 writesame 0x8d86281 thru 0x8d8c2f2 data_size 0xf(0x6072 bytes) 2024-09-16T22:16:20.823 INFO:teuthology.orchestra.run.smithi031.stdout:102 clone 2 order 22 su 131072 sc 15 2024-09-16T22:16:20.829 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:20.844 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:20.846+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:20.846 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:20.846 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:20.863 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:20.880 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:20.894 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:20.894 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:20.897 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:20.897 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:20.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:20.948 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:20.950+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:21.010 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:21.010 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:21.046 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:21.047+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:21.146 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:21.148+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:21.168 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:21.169 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:21.247 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:21.249+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:21.293 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.293 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:21.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1611039649999997, 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075317699999999995, 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133778+0000", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133779+0000", 2024-09-16T22:16:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3900000000000001e-07 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133782+0000", 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6689999999999999e-06 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133793+0000", 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0499000000000001e-05 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133801+0000", 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7989999999999994e-06 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133864+0000", 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799000000000004e-05 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.133887+0000", 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2289e-05 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.134492+0000", 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060517100000000005 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:17.134532+0000", 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712000000000002e-05 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:545 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-16T22:16:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:20.785348+0000", 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.509533876, 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022981600000000001, 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 545 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785348+0000", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785348+0000", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785350+0000", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.561e-06 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785358+0000", 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9039999999999992e-06 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785374+0000", 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6504999999999999e-05 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785381+0000", 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9480000000000002e-06 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785452+0000", 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0759000000000005e-05 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.785578+0000", 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012613899999999999 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:573 3.13 e27/23 3:cb9fc4f7:::rbd_data.11cdb0e83f6a.0000000000000238:head v 27'7, mlcod=27'5)", 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:20.822170+0000", 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47271204700000002, 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083610599999999998, 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822170+0000", 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822170+0000", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822171+0000", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018e-06 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822190+0000", 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8729999999999999e-05 2024-09-16T22:16:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822202+0000", 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2432000000000001e-05 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822209+0000", 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822267+0000", 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8241e-05 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822292+0000", 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4394000000000001e-05 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.822970+0000", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067864599999999998 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.823006+0000", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5868000000000003e-05 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:574 3.e e27/23 3:727894e8:::rbd_data.11cdb0e83f6a.0000000000000239:head v 27'5, mlcod=27'3)", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:20.825284+0000", 2024-09-16T22:16:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46959843400000001, 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00090821700000000001, 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825284+0000", 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825284+0000", 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825285+0000", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3900000000000003e-07 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825302+0000", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7609e-05 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825316+0000", 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3760999999999999e-05 2024-09-16T22:16:21.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825324+0000", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3490000000000003e-06 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825383+0000", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9868999999999999e-05 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.825407+0000", 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3416000000000001e-05 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.826148+0000", 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074165199999999998 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:20.826192+0000", 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3621999999999997e-05 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:613 3.1c e28/23 3:3831950c:::rbd_directory:head v 28'9, mlcod=27'7)", 2024-09-16T22:16:21.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.171365+0000", 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.12351764, 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071214900000000003, 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171365+0000", 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171365+0000", 2024-09-16T22:16:21.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171365+0000", 2024-09-16T22:16:21.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6800000000000002e-07 2024-09-16T22:16:21.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171389+0000", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3611000000000001e-05 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171400+0000", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1549000000000001e-05 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171410+0000", 2024-09-16T22:16:21.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171478+0000", 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7197000000000001e-05 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.171504+0000", 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5918000000000002e-05 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.172043+0000", 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053959199999999998 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.172077+0000", 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3680999999999998e-05 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:617 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.178085+0000", 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.116797713, 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016743159999999999, 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 617 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178085+0000", 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178085+0000", 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178087+0000", 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0829999999999998e-06 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178094+0000", 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1199999999999996e-06 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178110+0000", 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6288999999999999e-05 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178117+0000", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0670000000000003e-06 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178170+0000", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2673000000000002e-05 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178263+0000", 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3356000000000005e-05 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.178400+0000", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013668799999999999 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179092+0000", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069221500000000004 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179693+0000", 2024-09-16T22:16:21.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060108099999999995 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179730+0000", 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6853000000000001e-05 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179759+0000", 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8890999999999998e-05 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4557.0:617 3.15 e28/23 ondisk, result = 0)", 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.179608+0000", 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.11527478200000001, 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155828, 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179608+0000", 2024-09-16T22:16:21.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179608+0000", 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179608+0000", 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6599999999999994e-07 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179613+0000", 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7550000000000002e-06 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179627+0000", 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3732e-05 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179635+0000", 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3699999999999995e-06 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179682+0000", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7108999999999999e-05 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179688+0000", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8950000000000003e-06 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.179763+0000", 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5000999999999996e-05 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.330 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:21.338 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T22:16:21.348 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:21.349+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:21.449 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:21.451+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:21.450 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:21.450 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:21.550 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:21.552+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:21.588 INFO:teuthology.orchestra.run.smithi031.stdout:103 read 0x679d7d7 thru 0x67a8fa5 (0xb7cf bytes) 2024-09-16T22:16:21.588 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.588 INFO:teuthology.orchestra.run.smithi031.stdout:107 writesame 0x1e50e9c thru 0x1e56813 data_size 0x1c(0x5978 bytes) 2024-09-16T22:16:21.611 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.611 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:21.611 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:21.611 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:21.611 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.612 INFO:teuthology.orchestra.run.smithi031.stdout:108 trunc from 0xdcc503f to 0x4a73ad9 2024-09-16T22:16:21.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:21.647 INFO:teuthology.orchestra.run.smithi031.stdout:109 write 0x5c927a8 thru 0x5c9459c (0x1df5 bytes) 2024-09-16T22:16:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:110 trunc from 0x5c9459d to 0x4e7bd53 2024-09-16T22:16:21.651 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:21.652+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:21.677 INFO:teuthology.orchestra.run.smithi031.stdout:111 write 0xe0dbe92 thru 0xe0e26ac (0x681b bytes) 2024-09-16T22:16:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:112 punch from 0x6e4fcfb to 0x6e570b7, (0x73bc bytes) 2024-09-16T22:16:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:113 read 0x5582948 thru 0x5583f6b (0x1624 bytes) 2024-09-16T22:16:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:114 read 0x8e7b92c thru 0x8e84b45 (0x921a bytes) 2024-09-16T22:16:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:115 read 0xba860cb thru 0xba86995 (0x8cb bytes) 2024-09-16T22:16:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:117 punch from 0x1e7fb28 to 0x1e8756d, (0x7a45 bytes) 2024-09-16T22:16:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:118 trunc from 0xe0e26ad to 0x74a8096 2024-09-16T22:16:21.706 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.706 INFO:teuthology.orchestra.run.smithi031.stdout:122 punch from 0x4c497f3 to 0x4c4faa1, (0x62ae bytes) 2024-09-16T22:16:21.706 INFO:teuthology.orchestra.run.smithi031.stdout:123 write 0xbce90 thru 0xc852a (0xb69b bytes) 2024-09-16T22:16:21.711 INFO:teuthology.orchestra.run.smithi031.stdout:124 punch from 0x6e1b537 to 0x6e23b6b, (0x8634 bytes) 2024-09-16T22:16:21.712 INFO:teuthology.orchestra.run.smithi031.stdout:125 write 0x17b0960 thru 0x17b6d6f (0x6410 bytes) 2024-09-16T22:16:21.718 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.731 INFO:teuthology.orchestra.run.smithi031.stdout:131 writesame 0xdff57b4 thru 0xe0003be data_size 0x35(0xac0b bytes) 2024-09-16T22:16:21.731 INFO:teuthology.orchestra.run.smithi031.stdout:132 writesame 0x3da599c thru 0x3dabdfb data_size 0x58(0x6460 bytes) 2024-09-16T22:16:21.732 INFO:teuthology.orchestra.run.smithi031.stdout:133 write 0x2ab946c thru 0x2ac2b2d (0x96c2 bytes) 2024-09-16T22:16:21.737 INFO:teuthology.orchestra.run.smithi031.stdout:134 punch from 0x37510f4 to 0x3757364, (0x6270 bytes) 2024-09-16T22:16:21.737 INFO:teuthology.orchestra.run.smithi031.stdout:135 read 0xd7ebaf1 thru 0xd7ede23 (0x2333 bytes) 2024-09-16T22:16:21.737 INFO:teuthology.orchestra.run.smithi031.stdout:136 read 0xa86d93e thru 0xa872357 (0x4a1a bytes) 2024-09-16T22:16:21.737 INFO:teuthology.orchestra.run.smithi031.stdout:137 write 0xb31246a thru 0xb3214ec (0xf083 bytes) 2024-09-16T22:16:21.751 INFO:teuthology.orchestra.run.smithi031.stdout:138 trunc from 0xe0003bf to 0x62a4cad 2024-09-16T22:16:21.751 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:21.753+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:21.752 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.361334420999999, 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015694280000000001, 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391488+0000", 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6279999999999999e-06 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391496+0000", 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9119999999999998e-06 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391507+0000", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0251000000000001e-05 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391515+0000", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1890000000000007e-06 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391562+0000", 2024-09-16T22:16:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7051000000000001e-05 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391570+0000", 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1899999999999995e-06 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.401974+0000", 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010404096999999999 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402150+0000", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017572 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402318+0000", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016847499999999999 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402960+0000", 2024-09-16T22:16:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064145900000000004 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407117+0000", 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041574799999999999 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407153+0000", 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5587000000000003e-05 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407181+0000", 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8240999999999999e-05 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222694301000001, 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077487449999999996, 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530130+0000", 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530135+0000", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6659999999999996e-06 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530146+0000", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0379e-05 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531751+0000", 2024-09-16T22:16:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001605022 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531840+0000", 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9159000000000005e-05 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531986+0000", 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000145978 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534511+0000", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025251330000000002 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537823+0000", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033120950000000001 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537855+0000", 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1955999999999997e-05 2024-09-16T22:16:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537876+0000", 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0789999999999999e-05 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222529381000001, 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007677835, 2024-09-16T22:16:21.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 43 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530299+0000", 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530304+0000", 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530307+0000", 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.095e-06 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532608+0000", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023004319999999998 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532653+0000", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5506000000000001e-05 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532734+0000", 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0937000000000001e-05 2024-09-16T22:16:21.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534517+0000", 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001782596 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537905+0000", 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033881570000000001 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537945+0000", 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0624000000000003e-05 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537970+0000", 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4216999999999999e-05 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222450724000002, 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076823799999999999, 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 44 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:16:21.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530444+0000", 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2640999999999998e-05 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530451+0000", 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530455+0000", 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0690000000000003e-06 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533032+0000", 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025771930000000002 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533083+0000", 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1218999999999999e-05 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533162+0000", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8898000000000003e-05 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535469+0000", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023075109999999999 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538005+0000", 2024-09-16T22:16:21.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002535388 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538032+0000", 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6954e-05 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538053+0000", 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1080000000000001e-05 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222359862000001, 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081949469999999993, 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45 2024-09-16T22:16:21.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530462+0000", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530464+0000", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530468+0000", 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5410000000000001e-06 2024-09-16T22:16:21.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530471+0000", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533574+0000", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031031169999999999 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533631+0000", 2024-09-16T22:16:21.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533707+0000", 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6099000000000004e-05 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535480+0000", 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017729169999999999 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538616+0000", 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031358829999999999 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538639+0000", 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3315e-05 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538656+0000", 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7084000000000001e-05 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222345914999998, 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082594290000000004, 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 46 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530476+0000", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6899999999999999e-07 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530478+0000", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530481+0000", 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3689999999999998e-06 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530483+0000", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534007+0000", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035237409999999999 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534089+0000", 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534239+0000", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149615 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536005+0000", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017664099999999999 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538687+0000", 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026822949999999999 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538714+0000", 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7236000000000001e-05 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538735+0000", 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0293e-05 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222307372, 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082918049999999993, 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 47 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530608+0000", 2024-09-16T22:16:21.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4163e-05 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530614+0000", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530617+0000", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534562+0000", 2024-09-16T22:16:21.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039451900000000003 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534617+0000", 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5281999999999997e-05 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534695+0000", 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7344999999999999e-05 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536013+0000", 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013184970000000001 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538757+0000", 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027440049999999999 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538787+0000", 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9734e-05 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538806+0000", 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8678e-05 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222197810000001, 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010151530000000001, 2024-09-16T22:16:21.796 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 48 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530624+0000", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4400000000000001e-07 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530626+0000", 2024-09-16T22:16:21.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530629+0000", 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3519999999999998e-06 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530632+0000", 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535102+0000", 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044695769999999998 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535147+0000", 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5874999999999997e-05 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535225+0000", 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7262000000000007e-05 2024-09-16T22:16:21.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536945+0000", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001720297 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540728+0000", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003783472 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540754+0000", 2024-09-16T22:16:21.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5791000000000001e-05 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540775+0000", 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0733999999999998e-05 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:21.802 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222164921000001, 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010193045, 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:21.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530657+0000", 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530660+0000", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4180000000000001e-06 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530666+0000", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7370000000000003e-06 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530674+0000", 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0399999999999993e-06 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535615+0000", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049413870000000002 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535666+0000", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.092e-05 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535755+0000", 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8263999999999994e-05 2024-09-16T22:16:21.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536950+0000", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001195776 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540802+0000", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038521749999999998 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540828+0000", 2024-09-16T22:16:21.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5731000000000001e-05 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540849+0000", 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1182000000000001e-05 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.222082576999998, 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010330358, 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50 2024-09-16T22:16:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530828+0000", 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8504999999999994e-05 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530835+0000", 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7179999999999996e-06 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530839+0000", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536044+0000", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052045609999999999 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536092+0000", 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100999999999999e-05 2024-09-16T22:16:21.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536171+0000", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9357999999999995e-05 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536954+0000", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078211999999999997 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541028+0000", 2024-09-16T22:16:21.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040741529999999996 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541052+0000", 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4267e-05 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541069+0000", 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7088999999999999e-05 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.221974530000001, 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010291738999999999, 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51 2024-09-16T22:16:21.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:21.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.39e-07 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530849+0000", 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2639999999999998e-06 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530853+0000", 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5279999999999999e-06 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530856+0000", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536566+0000", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057098060000000004 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536617+0000", 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0711000000000003e-05 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536691+0000", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4536999999999995e-05 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537339+0000", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064814099999999997 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541092+0000", 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037528790000000002 2024-09-16T22:16:21.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541117+0000", 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5279999999999999e-05 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541138+0000", 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1047999999999999e-05 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.221961113999999, 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010348868000000001, 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 52 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:21.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530863+0000", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530867+0000", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530879+0000", 2024-09-16T22:16:21.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536976+0000", 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060963989999999997 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537037+0000", 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1623000000000003e-05 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537121+0000", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3033000000000003e-05 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537655+0000", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053447399999999995 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541165+0000", 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035098120000000002 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541190+0000", 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5313e-05 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541209+0000", 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8878000000000001e-05 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907942239, 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089954470000000002, 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 70 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0199999999999998e-07 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844882+0000", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844886+0000", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844889+0000", 2024-09-16T22:16:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848458+0000", 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035685589999999998 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848544+0000", 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6298000000000006e-05 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848675+0000", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130808 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851222+0000", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025471500000000002 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853828+0000", 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026053309999999998 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853852+0000", 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.495e-05 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853874+0000", 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1982999999999999e-05 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907927857000001, 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090279209999999995, 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 71 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844894+0000", 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:16:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844896+0000", 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.069e-06 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844899+0000", 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844902+0000", 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849147+0000", 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042455749999999997 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849225+0000", 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7844999999999998e-05 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849353+0000", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012841400000000001 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851381+0000", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020275390000000001 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853879+0000", 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024981949999999999 2024-09-16T22:16:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853900+0000", 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1138e-05 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853921+0000", 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1029000000000001e-05 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907857311000001, 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011701283, 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 72 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-07 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845021+0000", 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6480000000000001e-05 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845026+0000", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3449999999999996e-06 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845033+0000", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849957+0000", 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049242820000000003 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850000+0000", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3381999999999999e-05 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850076+0000", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853789+0000", 2024-09-16T22:16:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00371274 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856571+0000", 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002782432 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856623+0000", 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1907000000000003e-05 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856665+0000", 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2347e-05 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907781981999999, 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011700838, 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 73 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845040+0000", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845042+0000", 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:16:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845047+0000", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0860000000000001e-06 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845057+0000", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7149999999999993e-06 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851416+0000", 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063595580000000004 2024-09-16T22:16:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851460+0000", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510999999999997e-05 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851531+0000", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1266000000000005e-05 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853820+0000", 2024-09-16T22:16:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022893879999999998 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856673+0000", 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028527359999999998 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856720+0000", 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7027000000000002e-05 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856740+0000", 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9973999999999999e-05 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907756397, 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011717971000000001, 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 74 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845070+0000", 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000004e-06 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845074+0000", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9519999999999996e-06 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845076+0000", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000001e-06 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851800+0000", 2024-09-16T22:16:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067234740000000001 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851883+0000", 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2935999999999993e-05 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852017+0000", 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013409199999999999 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853945+0000", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019281179999999999 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856745+0000", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002800276 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856766+0000", 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0370999999999999e-05 2024-09-16T22:16:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856783+0000", 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7132000000000001e-05 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907740334, 2024-09-16T22:16:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014248581999999999, 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 75 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845140+0000", 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9061000000000003e-05 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845146+0000", 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845150+0000", 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5899999999999999e-06 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852357+0000", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072068239999999997 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852403+0000", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5924999999999998e-05 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852502+0000", 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9129000000000006e-05 2024-09-16T22:16:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854543+0000", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020415730000000001 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859274+0000", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047301799999999996 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859305+0000", 2024-09-16T22:16:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1117e-05 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859329+0000", 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4760999999999999e-05 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907662496, 2024-09-16T22:16:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014219621, 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 76 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845165+0000", 2024-09-16T22:16:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999998e-06 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845169+0000", 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9589999999999997e-06 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845172+0000", 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.709e-06 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853983+0000", 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088106799999999996 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854065+0000", 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2452000000000003e-05 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854200+0000", 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013444600000000001 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.855726+0000", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015263049999999999 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859334+0000", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036082890000000002 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859355+0000", 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0820999999999999e-05 2024-09-16T22:16:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859378+0000", 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3068e-05 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.623442231999999, 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071099739999999998, 2024-09-16T22:16:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 94 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129380+0000", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129383+0000", 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8850000000000001e-06 2024-09-16T22:16:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129390+0000", 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129396+0000", 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131718+0000", 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023225619999999998 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131759+0000", 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0601e-05 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131829+0000", 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9830000000000001e-05 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.135241+0000", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003412567 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136401+0000", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011601630000000001 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136452+0000", 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1113999999999999e-05 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136489+0000", 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6575999999999997e-05 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:21.875 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:21.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:139 writesame 0x3cef250 thru 0x3cf19ed data_size 0xb(0x279e bytes) 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:140 read 0x1e437c1 thru 0x1e4c799 (0x8fd9 bytes) 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:141 write 0xe2afc9e thru 0xe2b5cb0 (0x6013 bytes) 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:142 punch from 0x749828b to 0x749b17f, (0x2ef4 bytes) 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:148 write 0x8b7899a thru 0x8b7ebe5 (0x624c bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:149 read 0x673b9fe thru 0x673e7f5 (0x2df8 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:150 read 0x698a3df thru 0x698e6f4 (0x4316 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:151 write 0xd7fe3e3 thru 0xd80110f (0x2d2d bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:154 trunc from 0xe2b5cb1 to 0xdf3cdb9 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:155 writesame 0x7e6ba87 thru 0x7e6bfe6 data_size 0x10(0x560 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:157 read 0x6e09b00 thru 0x6e0fd9f (0x62a0 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:158 trunc from 0xdf3cdb9 to 0x81612ff 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:159 write 0x432596d thru 0x432acd4 (0x5368 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:160 writesame 0xbf4991c thru 0xbf5665d data_size 0x2f(0xcd42 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:163 writesame 0x5ee23c9 thru 0x5ee5a8b data_size 0x1(0x36c3 bytes) 2024-09-16T22:16:21.880 INFO:teuthology.orchestra.run.smithi031.stdout:164 read 0x2c7b811 thru 0x2c85b06 (0xa2f6 bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:165 write 0x1a3af1b thru 0x1a3fc1f (0x4d05 bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:166 writesame 0x6536b73 thru 0x653b82a data_size 0x8(0x4cb8 bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:168 write 0xe397c47 thru 0xe3a23ed (0xa7a7 bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:169 punch from 0x3b3ca40 to 0x3b4ba0b, (0xefcb bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:170 punch from 0xc5ef0db to 0xc5f17ea, (0x270f bytes) 2024-09-16T22:16:21.881 INFO:teuthology.orchestra.run.smithi031.stdout:171 trunc from 0xe3a23ee to 0x47f14e8 2024-09-16T22:16:21.881 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:21.878+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:21.890 INFO:teuthology.orchestra.run.smithi031.stdout:172 read 0x1a36a80 thru 0x1a3bc56 (0x51d7 bytes) 2024-09-16T22:16:21.891 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.908 INFO:teuthology.orchestra.run.smithi031.stdout:175 writesame 0xe0a1b45 thru 0xe0a97c9 data_size 0x1(0x7c85 bytes) 2024-09-16T22:16:21.908 INFO:teuthology.orchestra.run.smithi031.stdout:176 read 0xc567465 thru 0xc57568f (0xe22b bytes) 2024-09-16T22:16:21.912 INFO:teuthology.orchestra.run.smithi031.stdout:177 trunc from 0xe0a97ca to 0xa9619c9 2024-09-16T22:16:21.924 INFO:teuthology.orchestra.run.smithi031.stdout:178 writesame 0xac3ef6e thru 0xac4bbef data_size 0x1(0xcc82 bytes) 2024-09-16T22:16:21.930 INFO:teuthology.orchestra.run.smithi031.stdout:179 writesame 0xb3cb769 thru 0xb3d136b data_size 0x7(0x5c03 bytes) 2024-09-16T22:16:21.939 INFO:teuthology.orchestra.run.smithi031.stdout:180 write 0xd434100 thru 0xd440b94 (0xca95 bytes) 2024-09-16T22:16:21.943 INFO:teuthology.orchestra.run.smithi031.stdout:181 write 0xa7e734e thru 0xa7eb46a (0x411d bytes) 2024-09-16T22:16:21.947 INFO:teuthology.orchestra.run.smithi031.stdout:182 write 0x4792e5e thru 0x479ead1 (0xbc74 bytes) 2024-09-16T22:16:21.949 INFO:teuthology.orchestra.run.smithi031.stdout:183 punch from 0x34e87cd to 0x34ecd5a, (0x458d bytes) 2024-09-16T22:16:21.949 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:21.949 INFO:teuthology.orchestra.run.smithi031.stdout:185 read 0x2bb76f1 thru 0x2bbacb9 (0x35c9 bytes) 2024-09-16T22:16:21.949 INFO:teuthology.orchestra.run.smithi031.stdout:187 punch from 0xb0f30d1 to 0xb0f5f97, (0x2ec6 bytes) 2024-09-16T22:16:21.952 INFO:teuthology.orchestra.run.smithi031.stdout:189 trunc from 0xd440b95 to 0xa969428 2024-09-16T22:16:21.968 INFO:teuthology.orchestra.run.smithi031.stdout:190 writesame 0x8674e66 thru 0x867d8bc data_size 0xf(0x8a57 bytes) 2024-09-16T22:16:21.972 INFO:teuthology.orchestra.run.smithi031.stdout:191 write 0x2f9aa2f thru 0x2fa9314 (0xe8e6 bytes) 2024-09-16T22:16:21.977 INFO:teuthology.orchestra.run.smithi031.stdout:192 trunc from 0xa969428 to 0x8695564 2024-09-16T22:16:21.977 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:21.979+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:21.989 INFO:teuthology.orchestra.run.smithi031.stdout:193 trunc from 0x8695564 to 0xdbc8cc2 2024-09-16T22:16:21.995 INFO:teuthology.orchestra.run.smithi031.stdout:195 write 0x846412c thru 0x8473222 (0xf0f7 bytes) 2024-09-16T22:16:21.999 INFO:teuthology.orchestra.run.smithi031.stdout:196 write 0xda72fe3 thru 0xda78422 (0x5440 bytes) 2024-09-16T22:16:22.003 INFO:teuthology.orchestra.run.smithi031.stdout:199 write 0x8f9b271 thru 0x8fa8def (0xdb7f bytes) 2024-09-16T22:16:22.007 INFO:teuthology.orchestra.run.smithi031.stdout:201 write 0x47b1d5f thru 0x47b410c (0x23ae bytes) 2024-09-16T22:16:22.013 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:22.013 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.014 INFO:teuthology.orchestra.run.smithi031.stdout:203 writesame 0xe41e07d thru 0xe4296be data_size 0x29(0xb642 bytes) 2024-09-16T22:16:22.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.020 INFO:teuthology.orchestra.run.smithi031.stdout:206 writesame 0xe48527b thru 0xe49329d data_size 0x7(0xe023 bytes) 2024-09-16T22:16:22.020 INFO:teuthology.orchestra.run.smithi031.stdout:208 write 0xa9bfa5f thru 0xa9c19bd (0x1f5f bytes) 2024-09-16T22:16:22.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:22.023 INFO:teuthology.orchestra.run.smithi031.stdout:210 write 0x4895cc3 thru 0x489803c (0x237a bytes) 2024-09-16T22:16:22.032 INFO:teuthology.orchestra.run.smithi031.stdout:211 trunc from 0xe49329e to 0x76e059c 2024-09-16T22:16:22.058 INFO:teuthology.orchestra.run.smithi031.stdout:212 read 0x5e01478 thru 0x5e01705 (0x28e bytes) 2024-09-16T22:16:22.059 INFO:teuthology.orchestra.run.smithi031.stdout:213 read 0x1dd7412 thru 0x1dda0d9 (0x2cc8 bytes) 2024-09-16T22:16:22.059 INFO:teuthology.orchestra.run.smithi031.stdout:214 write 0x539b621 thru 0x53a8f80 (0xd960 bytes) 2024-09-16T22:16:22.063 INFO:teuthology.orchestra.run.smithi031.stdout:215 read 0xe686cc thru 0xe6dae1 (0x5416 bytes) 2024-09-16T22:16:22.063 INFO:teuthology.orchestra.run.smithi031.stdout:217 read 0x6842119 thru 0x68485cd (0x64b5 bytes) 2024-09-16T22:16:22.074 INFO:teuthology.orchestra.run.smithi031.stdout:218 write 0x9dd89f6 thru 0x9ddb18b (0x2796 bytes) 2024-09-16T22:16:22.078 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:22.079+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:22.082 INFO:teuthology.orchestra.run.smithi031.stdout:221 trunc from 0x9ddb18c to 0x7e450b7 2024-09-16T22:16:22.093 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.093 INFO:teuthology.orchestra.run.smithi031.stdout:223 writesame 0xa3db6f thru 0xa432ce data_size 0x30(0x5760 bytes) 2024-09-16T22:16:22.105 INFO:teuthology.orchestra.run.smithi031.stdout:224 trunc from 0x7e450b7 to 0x3819f9e 2024-09-16T22:16:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:226 read 0x1dd3ce8 thru 0x1de1805 (0xdb1e bytes) 2024-09-16T22:16:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:227 punch from 0x1fb08f0 to 0x1fb105d, (0x76d bytes) 2024-09-16T22:16:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:228 punch from 0x2d93263 to 0x2d9787d, (0x461a bytes) 2024-09-16T22:16:22.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:16:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:229 trunc from 0x3819f9e to 0xd4fe225 2024-09-16T22:16:22.161 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.166 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:22.166 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.175 INFO:teuthology.orchestra.run.smithi031.stdout:233 trunc from 0xd4fe225 to 0x18bc49c 2024-09-16T22:16:22.179 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:22.180+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:22.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:22.193 INFO:teuthology.orchestra.run.smithi031.stdout:234 write 0x1b06bed thru 0x1b0df89 (0x739d bytes) 2024-09-16T22:16:22.204 INFO:teuthology.orchestra.run.smithi031.stdout:235 trunc from 0x1b0df8a to 0xa3e403b 2024-09-16T22:16:22.210 INFO:teuthology.orchestra.run.smithi031.stdout:236 writesame 0x617a25b thru 0x61862fc data_size 0x3(0xc0a2 bytes) 2024-09-16T22:16:22.218 INFO:teuthology.orchestra.run.smithi031.stdout:237 write 0x98e6566 thru 0x98f4ffb (0xea96 bytes) 2024-09-16T22:16:22.222 INFO:teuthology.orchestra.run.smithi031.stdout:238 write 0xfcbff thru 0x104b95 (0x7f97 bytes) 2024-09-16T22:16:22.223 INFO:teuthology.orchestra.run.smithi031.stdout:239 writesame 0x64ee52f thru 0x64f04fa data_size 0x2c(0x1fcc bytes) 2024-09-16T22:16:22.227 INFO:teuthology.orchestra.run.smithi031.stdout:240 read 0x6e9ead7 thru 0x6eae968 (0xfe92 bytes) 2024-09-16T22:16:22.227 INFO:teuthology.orchestra.run.smithi031.stdout:241 read 0x7daadc2 thru 0x7db418e (0x93cd bytes) 2024-09-16T22:16:22.228 INFO:teuthology.orchestra.run.smithi031.stdout:242 writesame 0x621abaa thru 0x621d0b9 data_size 0x8(0x2510 bytes) 2024-09-16T22:16:22.231 INFO:teuthology.orchestra.run.smithi031.stdout:243 read 0x204942f thru 0x2058004 (0xebd6 bytes) 2024-09-16T22:16:22.232 INFO:teuthology.orchestra.run.smithi031.stdout:245 punch from 0x911dea8 to 0x912bd86, (0xdede bytes) 2024-09-16T22:16:22.232 INFO:teuthology.orchestra.run.smithi031.stdout:247 read 0x3bf9dd thru 0x3cccd4 (0xd2f8 bytes) 2024-09-16T22:16:22.239 INFO:teuthology.orchestra.run.smithi031.stdout:248 trunc from 0xa3e403b to 0x4002a53 2024-09-16T22:16:22.251 INFO:teuthology.orchestra.run.smithi031.stdout:249 read 0x31fa303 thru 0x31fc3c9 (0x20c7 bytes) 2024-09-16T22:16:22.252 INFO:teuthology.orchestra.run.smithi031.stdout:250 punch from 0x19d08a2 to 0x19d2753, (0x1eb1 bytes) 2024-09-16T22:16:22.268 INFO:teuthology.orchestra.run.smithi031.stdout:252 write 0xc4bfe99 thru 0xc4cb2ad (0xb415 bytes) 2024-09-16T22:16:22.279 INFO:teuthology.orchestra.run.smithi031.stdout:253 writesame 0xcf2e426 thru 0xcf3097e data_size 0x1(0x2559 bytes) 2024-09-16T22:16:22.279 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:22.281+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:22.287 INFO:teuthology.orchestra.run.smithi031.stdout:254 write 0xed54218 thru 0xed54c28 (0xa11 bytes) 2024-09-16T22:16:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:255 write 0x412f02f thru 0x413b3ec (0xc3be bytes) 2024-09-16T22:16:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:256 punch from 0x996ac1b to 0x9979e41, (0xf226 bytes) 2024-09-16T22:16:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:259 writesame 0x4848b95 thru 0x4856378 data_size 0x52(0xd7e4 bytes) 2024-09-16T22:16:22.296 INFO:teuthology.orchestra.run.smithi031.stdout:260 flatten 2024-09-16T22:16:22.302 INFO:teuthology.orchestra.run.smithi031.stdout:261 read 0x47d289d thru 0x47d3ea3 (0x1607 bytes) 2024-09-16T22:16:22.303 INFO:teuthology.orchestra.run.smithi031.stdout:262 write 0x198c527 thru 0x1999305 (0xcddf bytes) 2024-09-16T22:16:22.305 INFO:teuthology.orchestra.run.smithi031.stdout:263 read 0x2cd9e2d thru 0x2ce9a88 (0xfc5c bytes) 2024-09-16T22:16:22.305 INFO:teuthology.orchestra.run.smithi031.stdout:264 writesame 0x7b91c2f thru 0x7ba0945 data_size 0x3d(0xed17 bytes) 2024-09-16T22:16:22.310 INFO:teuthology.orchestra.run.smithi031.stdout:265 punch from 0x5b6bb36 to 0x5b79033, (0xd4fd bytes) 2024-09-16T22:16:22.310 INFO:teuthology.orchestra.run.smithi031.stdout:267 writesame 0xe2617ac thru 0xe268bf3 data_size 0x3d(0x7448 bytes) 2024-09-16T22:16:22.322 INFO:teuthology.orchestra.run.smithi031.stdout:269 trunc from 0xed54c29 to 0x88e8727 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:814 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.629680+0000", 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.695807018, 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003591519, 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 814 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629680+0000", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629680+0000", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629683+0000", 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:16:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629691+0000", 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.687e-06 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629711+0000", 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.967e-05 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629722+0000", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1046000000000001e-05 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629807+0000", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5044999999999996e-05 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629967+0000", 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159646 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630156+0000", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189147 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.631828+0000", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001671994 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.633172+0000", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001344472 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.633228+0000", 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6189e-05 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.633272+0000", 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3460000000000001e-05 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:890 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [stat,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.698259+0000", 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62722848799999997, 2024-09-16T22:16:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043994480000000002, 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 890 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698259+0000", 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698259+0000", 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698261+0000", 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698267+0000", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698286+0000", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8615e-05 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698294+0000", 2024-09-16T22:16:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0039999999999999e-06 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698362+0000", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8073999999999999e-05 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698455+0000", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.698607+0000", 2024-09-16T22:16:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000152645 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699598+0000", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099074499999999991 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702571+0000", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029727130000000001 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702626+0000", 2024-09-16T22:16:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5413e-05 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702658+0000", 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2125000000000002e-05 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:896 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.699086+0000", 2024-09-16T22:16:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62640177699999999, 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036770240000000001, 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 896 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699086+0000", 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699086+0000", 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699087+0000", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.065e-06 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699091+0000", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3950000000000003e-06 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699107+0000", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5916000000000001e-05 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699115+0000", 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7470000000000005e-06 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699187+0000", 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2760000000000001e-05 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699270+0000", 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2553999999999996e-05 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.699395+0000", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012447599999999999 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.700031+0000", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063621399999999996 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702691+0000", 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002659799 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702726+0000", 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5203999999999997e-05 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.702763+0000", 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6894000000000003e-05 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:938 3.e 3:73158de0:::rbd_data.11cd70c2eac3.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1640800~25616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.716695+0000", 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.60879259799999996, 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040201910000000002, 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 938 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716695+0000", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716695+0000", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716696+0000", 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.525e-06 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716745+0000", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9141000000000003e-05 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716758+0000", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2169e-05 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716764+0000", 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0689999999999998e-06 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716811+0000", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7651999999999997e-05 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.716903+0000", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1902000000000002e-05 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.717060+0000", 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015662200000000001 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.720421+0000", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033612009999999999 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.720638+0000", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000217316 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.720674+0000", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6015e-05 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.720715+0000", 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0578999999999999e-05 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:986 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.758567+0000", 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.56692023499999999, 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043827060000000001, 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 986 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758567+0000", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758567+0000", 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758568+0000", 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4300000000000002e-07 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758572+0000", 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5750000000000001e-06 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758581+0000", 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3850000000000008e-06 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758587+0000", 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3600000000000001e-06 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758803+0000", 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021546900000000001 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758882+0000", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8875000000000006e-05 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.759010+0000", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012804599999999999 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.760377+0000", 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013669909999999999 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762868+0000", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024910760000000001 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762901+0000", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3229000000000003e-05 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762950+0000", 2024-09-16T22:16:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8857000000000003e-05 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1095 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [stat,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.444439537, 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063311330000000001, 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1095 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881049+0000", 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5680000000000001e-06 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881056+0000", 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3430000000000001e-06 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881071+0000", 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4824000000000001e-05 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881079+0000", 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2400000000000007e-06 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881145+0000", 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6544999999999994e-05 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881231+0000", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6015999999999998e-05 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881363+0000", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013146499999999999 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882137+0000", 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077436300000000003 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887295+0000", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051576399999999998 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887330+0000", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5369e-05 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887379+0000", 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8760000000000001e-05 2024-09-16T22:16:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1101 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.44388829800000001, 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005882254, 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1101 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881600+0000", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000004e-07 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881604+0000", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8079999999999998e-06 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881614+0000", 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0451e-05 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881621+0000", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0829999999999998e-06 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881671+0000", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9738999999999999e-05 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881747+0000", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5634000000000001e-05 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881868+0000", 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121123 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.884524+0000", 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026559230000000001 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887415+0000", 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028909529999999999 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887443+0000", 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8844e-05 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887481+0000", 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7926000000000002e-05 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1097 3.3 e28/23 3:c55feef4:::rbd_data.11cd70c2eac3.0000000000000019:head v 28'8, mlcod=28'6)", 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.882189+0000", 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.44329832800000002, 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005030489, 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882189+0000", 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882189+0000", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882190+0000", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7899999999999998e-07 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882195+0000", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8230000000000002e-06 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882204+0000", 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6390000000000004e-06 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882210+0000", 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2580000000000001e-06 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882269+0000", 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8334000000000001e-05 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882291+0000", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1988e-05 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887175+0000", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048842670000000003 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887220+0000", 2024-09-16T22:16:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4501e-05 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1236 3.16 3:6e3b6ee1:::rbd_data.11cd70c2eac3.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.036395+0000", 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.28909233400000001, 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036445679999999999, 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1236 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036395+0000", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036395+0000", 2024-09-16T22:16:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036398+0000", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036406+0000", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9810000000000003e-06 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036425+0000", 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9738999999999999e-05 2024-09-16T22:16:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036435+0000", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9310000000000003e-06 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036493+0000", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7615000000000003e-05 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036552+0000", 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9197000000000003e-05 2024-09-16T22:16:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.036620+0000", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8141999999999999e-05 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.037276+0000", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065542200000000001 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.039966+0000", 2024-09-16T22:16:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026902670000000001 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.040007+0000", 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0571999999999997e-05 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.040040+0000", 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3092000000000001e-05 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1240 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [stat,truncate 4063232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.27964125899999998, 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00646095, 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1240 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045847+0000", 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1170000000000001e-06 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045853+0000", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045870+0000", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.715e-05 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045879+0000", 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7350000000000002e-06 2024-09-16T22:16:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045950+0000", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1886999999999998e-05 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046034+0000", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-05 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046175+0000", 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014056600000000001 2024-09-16T22:16:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047145+0000", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00097021599999999998 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052211+0000", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050656750000000004 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052262+0000", 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660999999999999e-05 2024-09-16T22:16:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052307+0000", 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4718000000000001e-05 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1246 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 4063232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.27903222700000002, 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006004995, 2024-09-16T22:16:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1246 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046456+0000", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999998e-07 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046460+0000", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0350000000000003e-06 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046472+0000", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2167000000000001e-05 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046479+0000", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046528+0000", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8717999999999997e-05 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046606+0000", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8794999999999994e-05 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046748+0000", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014167699999999999 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.049628+0000", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028798019999999999 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052360+0000", 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027316020000000001 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052417+0000", 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7003999999999998e-05 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052460+0000", 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3572000000000002e-05 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1242 3.3 e28/23 3:c55feef4:::rbd_data.11cd70c2eac3.0000000000000019:head v 28'11, mlcod=28'9)", 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.047016+0000", 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.27847112499999999, 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051904830000000001, 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047016+0000", 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047016+0000", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047017+0000", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000004e-07 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047023+0000", 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3439999999999997e-06 2024-09-16T22:16:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047033+0000", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7710000000000001e-06 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047040+0000", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6660000000000002e-06 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047104+0000", 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4090000000000005e-05 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047126+0000", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-05 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052163+0000", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005036797 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052207+0000", 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4094999999999997e-05 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1247 3.11 e28/23 3:8841aec0:::rbd_header.11cd70c2eac3:head v 28'43, mlcod=28'41)", 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.053079+0000", 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.272408443, 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042108400000000004, 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053079+0000", 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053079+0000", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053080+0000", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053087+0000", 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0720000000000002e-06 2024-09-16T22:16:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053099+0000", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2632999999999999e-05 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053105+0000", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8989999999999998e-06 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053157+0000", 2024-09-16T22:16:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1533000000000003e-05 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053171+0000", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4392e-05 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.057245+0000", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040741850000000001 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.057290+0000", 2024-09-16T22:16:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4403000000000002e-05 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1354 3.e 3:73158de0:::rbd_data.11cd70c2eac3.0000000000000009:head [stat,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.123221+0000", 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.20226671800000001, 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033417339999999999, 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1354 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123221+0000", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123221+0000", 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123221+0000", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4300000000000003e-07 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123224+0000", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0029999999999999e-06 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123233+0000", 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6379999999999993e-06 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123239+0000", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0680000000000002e-06 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123293+0000", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4203000000000003e-05 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123375+0000", 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1625999999999994e-05 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.123508+0000", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013312600000000001 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124784+0000", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012766069999999999 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126505+0000", 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017202090000000001 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126537+0000", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2489e-05 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126562+0000", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5222e-05 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1407 3.e 3:73158de0:::rbd_data.11cd70c2eac3.0000000000000009:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.178013+0000", 2024-09-16T22:16:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.147474309, 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046299210000000004, 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1407 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178013+0000", 2024-09-16T22:16:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178013+0000", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178016+0000", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6089999999999999e-06 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178025+0000", 2024-09-16T22:16:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1849999999999993e-06 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178048+0000", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3003999999999999e-05 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178059+0000", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0971e-05 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178130+0000", 2024-09-16T22:16:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0947999999999999e-05 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178217+0000", 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7059999999999997e-05 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178344+0000", 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012673599999999999 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.181963+0000", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003619349 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182541+0000", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00057841999999999995 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182597+0000", 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5801000000000003e-05 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182643+0000", 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5837999999999997e-05 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1410 3.16 3:6b0dcb7f:::rbd_data.11cd70c2eac3.000000000000000c:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.178143+0000", 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.147344001, 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046274159999999996, 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1410 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178143+0000", 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178143+0000", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178144+0000", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0139999999999999e-06 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178149+0000", 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-09-16T22:16:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178159+0000", 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0397000000000001e-05 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178167+0000", 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5100000000000001e-06 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178225+0000", 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8421000000000002e-05 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178295+0000", 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9937999999999993e-05 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178407+0000", 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111444 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.181994+0000", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035875379999999999 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182684+0000", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068968499999999999 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182733+0000", 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9116999999999997e-05 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182771+0000", 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7774999999999997e-05 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1404 3.3 e28/23 3:c49d5d91:::rbd_data.11cd70c2eac3.0000000000000006:head v 28'15, mlcod=28'13)", 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.178607+0000", 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14688006100000001, 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034519080000000001, 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178607+0000", 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178607+0000", 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178608+0000", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999998e-07 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178615+0000", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178624+0000", 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3519999999999996e-06 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178630+0000", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178692+0000", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2170999999999998e-05 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178715+0000", 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2957000000000001e-05 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182006+0000", 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032910579999999999 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182059+0000", 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2719999999999997e-05 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1402 3.14 e28/23 3:287671ef:::rbd_data.11cd70c2eac3.0000000000000004:head v 28'8, mlcod=28'6)", 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179582+0000", 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14590529699999999, 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052275189999999999, 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179582+0000", 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179582+0000", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179583+0000", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3300000000000001e-07 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179589+0000", 2024-09-16T22:16:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179599+0000", 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.061e-05 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179605+0000", 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0419999999999999e-06 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179660+0000", 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4339000000000003e-05 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179677+0000", 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7501000000000001e-05 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.184760+0000", 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050822250000000001 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.184810+0000", 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0068999999999998e-05 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:1408 3.14 e28/23 3:2bf0f6b0:::rbd_data.11cd70c2eac3.000000000000000a:head v 28'9, mlcod=28'6)", 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.180580+0000", 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14490779000000001, 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042576890000000003, 2024-09-16T22:16:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180580+0000", 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180580+0000", 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180580+0000", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5899999999999995e-07 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180588+0000", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2509999999999997e-06 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180601+0000", 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3798000000000001e-05 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180609+0000", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1400000000000002e-06 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180655+0000", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5961999999999998e-05 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180675+0000", 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0256999999999999e-05 2024-09-16T22:16:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.184813+0000", 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041381079999999997 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.184837+0000", 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4314000000000001e-05 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1444 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.215873+0000", 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.109614051, 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046300270000000001, 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1444 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215873+0000", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215873+0000", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215876+0000", 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7049999999999999e-06 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215886+0000", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329999999999999e-05 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215909+0000", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2903000000000001e-05 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215925+0000", 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5957e-05 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215986+0000", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0306999999999997e-05 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.216034+0000", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8427000000000002e-05 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.216652+0000", 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061808999999999996 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.220412+0000", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003759919 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.220443+0000", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0707000000000003e-05 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.220476+0000", 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2981000000000002e-05 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.220503+0000", 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7701000000000001e-05 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.435 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.436 INFO:teuthology.orchestra.run.smithi031.stdout:270 write 0x7205b46 thru 0x720f864 (0x9d1f bytes) 2024-09-16T22:16:22.436 INFO:teuthology.orchestra.run.smithi031.stdout:271 write 0x3a0b15c thru 0x3a0f729 (0x45ce bytes) 2024-09-16T22:16:22.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:272 trunc from 0x88e8727 to 0xa240e2 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:273 trunc from 0xa240e2 to 0x8a7ffee 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:274 write 0xbb0c62c thru 0xbb0f895 (0x326a bytes) 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:275 read 0xa7a9afe thru 0xa7aa62b (0xb2e bytes) 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:276 read 0xdb83b4 thru 0xdc12a3 (0x8ef0 bytes) 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:277 write 0x24c528 thru 0x24f781 (0x325a bytes) 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:278 write 0xd242731 thru 0xd24a490 (0x7d60 bytes) 2024-09-16T22:16:22.438 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.439 INFO:teuthology.orchestra.run.smithi031.stdout:280 trunc from 0xd24a491 to 0xb6b61f 2024-09-16T22:16:22.439 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:22.438+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:22.439 INFO:teuthology.orchestra.run.smithi031.stdout:281 punch from 0xab347d to 0xab403a, (0xbbd bytes) 2024-09-16T22:16:22.439 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.440 INFO:teuthology.orchestra.run.smithi031.stdout:283 punch from 0x60b06f to 0x61925f, (0xe1f0 bytes) 2024-09-16T22:16:22.440 INFO:teuthology.orchestra.run.smithi031.stdout:284 read 0x9a98a2 thru 0x9ac517 (0x2c76 bytes) 2024-09-16T22:16:22.445 INFO:teuthology.orchestra.run.smithi031.stdout:285 writesame 0x152a95f thru 0x152fffc data_size 0x1(0x569e bytes) 2024-09-16T22:16:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.466 INFO:teuthology.orchestra.run.smithi031.stdout:287 write 0xe114383 thru 0xe11dd32 (0x99b0 bytes) 2024-09-16T22:16:22.469 INFO:teuthology.orchestra.run.smithi031.stdout:288 writesame 0xbda79b7 thru 0xbdaa542 data_size 0x6(0x2b8c bytes) 2024-09-16T22:16:22.480 INFO:teuthology.orchestra.run.smithi031.stdout:289 trunc from 0xe11dd33 to 0x87475e5 2024-09-16T22:16:22.500 INFO:teuthology.orchestra.run.smithi031.stdout:290 read 0x873c782 thru 0x873d54f (0xdce bytes) 2024-09-16T22:16:22.515 INFO:teuthology.orchestra.run.smithi031.stdout:292 write 0xe32dbfc thru 0xe3380c5 (0xa4ca bytes) 2024-09-16T22:16:22.522 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.522 INFO:teuthology.orchestra.run.smithi031.stdout:294 read 0x166d37d thru 0x1671ed2 (0x4b56 bytes) 2024-09-16T22:16:22.522 INFO:teuthology.orchestra.run.smithi031.stdout:295 write 0x90a22a4 thru 0x90afe08 (0xdb65 bytes) 2024-09-16T22:16:22.526 INFO:teuthology.orchestra.run.smithi031.stdout:297 write 0x4f50f1c thru 0x4f53ac9 (0x2bae bytes) 2024-09-16T22:16:22.530 INFO:teuthology.orchestra.run.smithi031.stdout:298 punch from 0x4a4ace0 to 0x4a57ce6, (0xd006 bytes) 2024-09-16T22:16:22.530 INFO:teuthology.orchestra.run.smithi031.stdout:299 write 0x7f45e7 thru 0x7f9cd7 (0x56f1 bytes) 2024-09-16T22:16:22.537 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:22.539+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:22.537 INFO:teuthology.orchestra.run.smithi031.stdout:300 write 0xe7ea145 thru 0xe7f7d14 (0xdbd0 bytes) 2024-09-16T22:16:22.542 INFO:teuthology.orchestra.run.smithi031.stdout:301 write 0x29a9fbb thru 0x29aaac3 (0xb09 bytes) 2024-09-16T22:16:22.546 INFO:teuthology.orchestra.run.smithi031.stdout:302 trunc from 0xe7f7d15 to 0xc809978 2024-09-16T22:16:22.559 INFO:teuthology.orchestra.run.smithi031.stdout:303 trunc from 0xc809978 to 0xa4d2f7d 2024-09-16T22:16:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:22.564 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:22.581 INFO:teuthology.orchestra.run.smithi031.stdout:304 write 0x5d90372 thru 0x5d9ebdf (0xe86e bytes) 2024-09-16T22:16:22.587 INFO:teuthology.orchestra.run.smithi031.stdout:305 read 0x3af7741 thru 0x3afe0d2 (0x6992 bytes) 2024-09-16T22:16:22.588 INFO:teuthology.orchestra.run.smithi031.stdout:308 read 0x78deaf6 thru 0x78dfa53 (0xf5e bytes) 2024-09-16T22:16:22.588 INFO:teuthology.orchestra.run.smithi031.stdout:310 read 0x88457f7 thru 0x884684a (0x1054 bytes) 2024-09-16T22:16:22.588 INFO:teuthology.orchestra.run.smithi031.stdout:311 read 0x60db4a5 thru 0x60e6405 (0xaf61 bytes) 2024-09-16T22:16:22.588 INFO:teuthology.orchestra.run.smithi031.stdout:312 writesame 0x7f783ab thru 0x7f81873 data_size 0x1(0x94c9 bytes) 2024-09-16T22:16:22.604 INFO:teuthology.orchestra.run.smithi031.stdout:314 read 0x92819c7 thru 0x9290662 (0xec9c bytes) 2024-09-16T22:16:22.605 INFO:teuthology.orchestra.run.smithi031.stdout:315 writesame 0x53e1b13 thru 0x53ee8f4 data_size 0x92(0xcde2 bytes) 2024-09-16T22:16:22.608 INFO:teuthology.orchestra.run.smithi031.stdout:316 write 0x1e27ca7 thru 0x1e36d58 (0xf0b2 bytes) 2024-09-16T22:16:22.610 INFO:teuthology.orchestra.run.smithi031.stdout:319 read 0x74a4590 thru 0x74a5638 (0x10a9 bytes) 2024-09-16T22:16:22.610 INFO:teuthology.orchestra.run.smithi031.stdout:320 write 0x5e22773 thru 0x5e22968 (0x1f6 bytes) 2024-09-16T22:16:22.617 INFO:teuthology.orchestra.run.smithi031.stdout:323 trunc from 0xa4d2f7d to 0x647a90e 2024-09-16T22:16:22.631 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.631 INFO:teuthology.orchestra.run.smithi031.stdout:326 write 0x449a5ec thru 0x44a796f (0xd384 bytes) 2024-09-16T22:16:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:327 read 0x4b289cb thru 0x4b2fc4c (0x7282 bytes) 2024-09-16T22:16:22.638 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.638 INFO:teuthology.orchestra.run.smithi031.stdout:329 punch from 0x3efb864 to 0x3f04668, (0x8e04 bytes) 2024-09-16T22:16:22.638 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:22.640+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:22.651 INFO:teuthology.orchestra.run.smithi031.stdout:330 write 0xb8ecc13 thru 0xb8ed4ab (0x899 bytes) 2024-09-16T22:16:22.658 INFO:teuthology.orchestra.run.smithi031.stdout:331 read 0x5cba33b thru 0x5cbbb03 (0x17c9 bytes) 2024-09-16T22:16:22.658 INFO:teuthology.orchestra.run.smithi031.stdout:332 read 0x130b31e thru 0x1310f0a (0x5bed bytes) 2024-09-16T22:16:22.669 INFO:teuthology.orchestra.run.smithi031.stdout:333 write 0xdf36929 thru 0xdf391b1 (0x2889 bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:336 read 0xcfb7994 thru 0xcfbd91c (0x5f89 bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:338 read 0x796d476 thru 0x7974d21 (0x78ac bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:339 read 0xd1b6328 thru 0xd1bdb0f (0x77e8 bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:340 punch from 0xdd80895 to 0xdd83991, (0x30fc bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:342 read 0xd6062c7 thru 0xd60d05d (0x6d97 bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:343 punch from 0xc28c02b to 0xc2939f9, (0x79ce bytes) 2024-09-16T22:16:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:346 read 0xb7a302a thru 0xb7b094f (0xd926 bytes) 2024-09-16T22:16:22.677 INFO:teuthology.orchestra.run.smithi031.stdout:347 writesame 0xa79b60 thru 0xa82e8c data_size 0x13(0x932d bytes) 2024-09-16T22:16:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:348 writesame 0x78def86 thru 0x78ec03d data_size 0x4(0xd0b8 bytes) 2024-09-16T22:16:22.690 INFO:teuthology.orchestra.run.smithi031.stdout:349 read 0x2a68c23 thru 0x2a68c42 (0x20 bytes) 2024-09-16T22:16:22.691 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.691 INFO:teuthology.orchestra.run.smithi031.stdout:351 writesame 0x5fa30f5 thru 0x5fa61ab data_size 0x1(0x30b7 bytes) 2024-09-16T22:16:22.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.698 INFO:teuthology.orchestra.run.smithi031.stdout:355 read 0xb497a06 thru 0xb4a4da3 (0xd39e bytes) 2024-09-16T22:16:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:356 read 0x9665ce8 thru 0x966fd4c (0xa065 bytes) 2024-09-16T22:16:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xed3123b 2024-09-16T22:16:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:358 trunc from 0xdf391b2 to 0xed3123b 2024-09-16T22:16:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:22.700 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.704 INFO:teuthology.orchestra.run.smithi031.stdout:359 read 0x89e08a9 thru 0x89e49ab (0x4103 bytes) 2024-09-16T22:16:22.704 INFO:teuthology.orchestra.run.smithi031.stdout:360 read 0x69ead09 thru 0x69f769e (0xc996 bytes) 2024-09-16T22:16:22.704 INFO:teuthology.orchestra.run.smithi031.stdout:361 writesame 0x9f89e71 thru 0x9f8e656 data_size 0x1(0x47e6 bytes) 2024-09-16T22:16:22.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:22.715 INFO:teuthology.orchestra.run.smithi031.stdout:362 read 0xbb9b06d thru 0xbba58f1 (0xa885 bytes) 2024-09-16T22:16:22.716 INFO:teuthology.orchestra.run.smithi031.stdout:363 read 0xab14d2e thru 0xab2249a (0xd76d bytes) 2024-09-16T22:16:22.716 INFO:teuthology.orchestra.run.smithi031.stdout:364 writesame 0x7c42880 thru 0x7c48cb5 data_size 0x2(0x6436 bytes) 2024-09-16T22:16:22.724 INFO:teuthology.orchestra.run.smithi031.stdout:365 read 0xc912c0 thru 0xc9a9cd (0x970e bytes) 2024-09-16T22:16:22.724 INFO:teuthology.orchestra.run.smithi031.stdout:366 writesame 0x98da26 thru 0x98f2dd data_size 0x2(0x18b8 bytes) 2024-09-16T22:16:22.726 INFO:teuthology.orchestra.run.smithi031.stdout:367 write 0x4c8ebbc thru 0x4c95ff0 (0x7435 bytes) 2024-09-16T22:16:22.733 INFO:teuthology.orchestra.run.smithi031.stdout:368 writesame 0x596c75c thru 0x597132d data_size 0xf(0x4bd2 bytes) 2024-09-16T22:16:22.736 INFO:teuthology.orchestra.run.smithi031.stdout:369 write 0x9304dac thru 0x930e6f4 (0x9949 bytes) 2024-09-16T22:16:22.738 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.738 INFO:teuthology.orchestra.run.smithi031.stdout:371 read 0xe0e74ee thru 0xe0e86f5 (0x1208 bytes) 2024-09-16T22:16:22.738 INFO:teuthology.orchestra.run.smithi031.stdout:373 read 0x58d4fa thru 0x58e4fc (0x1003 bytes) 2024-09-16T22:16:22.738 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:22.740+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:22.738 INFO:teuthology.orchestra.run.smithi031.stdout:374 write 0x3852a21 thru 0x385ef66 (0xc546 bytes) 2024-09-16T22:16:22.740 INFO:teuthology.orchestra.run.smithi031.stdout:375 punch from 0x87a690c to 0x87b0bf1, (0xa2e5 bytes) 2024-09-16T22:16:22.740 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.744 INFO:teuthology.orchestra.run.smithi031.stdout:377 trunc from 0xed3123b to 0xcceab0a 2024-09-16T22:16:22.750 INFO:teuthology.orchestra.run.smithi031.stdout:380 punch from 0xa52e8b1 to 0xa5376bb, (0x8e0a bytes) 2024-09-16T22:16:22.750 INFO:teuthology.orchestra.run.smithi031.stdout:381 write 0x914a3a5 thru 0x91570eb (0xcd47 bytes) 2024-09-16T22:16:22.751 INFO:teuthology.orchestra.run.smithi031.stdout:382 read 0x5a93c7c thru 0x5a942b7 (0x63c bytes) 2024-09-16T22:16:22.758 INFO:teuthology.orchestra.run.smithi031.stdout:383 write 0xe13ec3a thru 0xe141cb7 (0x307e bytes) 2024-09-16T22:16:22.762 INFO:teuthology.orchestra.run.smithi031.stdout:384 write 0x9db0b95 thru 0x9db5a16 (0x4e82 bytes) 2024-09-16T22:16:22.763 INFO:teuthology.orchestra.run.smithi031.stdout:385 write 0xe01bfd6 thru 0xe02579b (0x97c6 bytes) 2024-09-16T22:16:22.768 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.782 INFO:teuthology.orchestra.run.smithi031.stdout:388 trunc from 0xe141cb8 to 0x138ea53 2024-09-16T22:16:22.804 INFO:teuthology.orchestra.run.smithi031.stdout:390 read 0xa23a98 thru 0xa247be (0xd27 bytes) 2024-09-16T22:16:22.825 INFO:teuthology.orchestra.run.smithi031.stdout:391 writesame 0xd8d7882 thru 0xd8d7b02 data_size 0x1(0x281 bytes) 2024-09-16T22:16:22.825 INFO:teuthology.orchestra.run.smithi031.stdout:392 write 0x9c40229 thru 0x9c4e4d8 (0xe2b0 bytes) 2024-09-16T22:16:22.828 INFO:teuthology.orchestra.run.smithi031.stdout:393 read 0x59f37f6 thru 0x59fe4b5 (0xacc0 bytes) 2024-09-16T22:16:22.828 INFO:teuthology.orchestra.run.smithi031.stdout:394 writesame 0x274fccb thru 0x2757b9d data_size 0x1(0x7ed3 bytes) 2024-09-16T22:16:22.839 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:22.841+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.420004+0000", 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.422418348000001, 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048988829999999997, 2024-09-16T22:16:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420004+0000", 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420004+0000", 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420008+0000", 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7689999999999998e-06 2024-09-16T22:16:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420015+0000", 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9469999999999997e-06 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420030+0000", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.509e-05 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420045+0000", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4561e-05 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420127+0000", 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:16:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.420135+0000", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9990000000000008e-06 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.422889+0000", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027544539999999999 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.423119+0000", 2024-09-16T22:16:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022979900000000001 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.423309+0000", 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019038200000000001 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.424066+0000", 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075658400000000003 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.424832+0000", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076654800000000003 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.424872+0000", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-05 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.424903+0000", 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0833000000000001e-05 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.414932416999999, 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:806 3.a 3:56bea48c:::rbd_data.11cd70c2eac3.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.621890+0000", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.220532607, 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055326860000000002, 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 806 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621890+0000", 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621890+0000", 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621892+0000", 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9690000000000001e-06 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621900+0000", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7840000000000004e-06 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621923+0000", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3541e-05 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.621933+0000", 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011e-05 2024-09-16T22:16:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622008+0000", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4350000000000005e-05 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622172+0000", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016384799999999999 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622390+0000", 2024-09-16T22:16:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000218617 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624336+0000", 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019459449999999999 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627337+0000", 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030008750000000001 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627394+0000", 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7358999999999997e-05 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627423+0000", 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8288000000000001e-05 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:844 3.d 3:b751b771:::rbd_data.11cd70c2eac3.0000000000000013:head [stat,zero 1310720~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.652970+0000", 2024-09-16T22:16:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.189452516, 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041560349999999998, 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 844 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.652970+0000", 2024-09-16T22:16:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.652970+0000", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.652971+0000", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3900000000000004e-07 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.652987+0000", 2024-09-16T22:16:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.624e-05 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.652997+0000", 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9590000000000007e-06 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.653003+0000", 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.653135+0000", 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131824 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.653219+0000", 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3980999999999994e-05 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.657126+0000", 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039070110000000002 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:972 3.a 3:56bea48c:::rbd_data.11cd70c2eac3.000000000000000f:head [stat,truncate 2752512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.757797+0000", 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0846257690000001, 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050336770000000003, 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 972 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757797+0000", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757797+0000", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757799+0000", 2024-09-16T22:16:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6360000000000001e-06 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757803+0000", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9679999999999997e-06 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757815+0000", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1861999999999999e-05 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757827+0000", 2024-09-16T22:16:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1542999999999999e-05 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757882+0000", 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.757971+0000", 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9164000000000002e-05 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.758116+0000", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014507300000000001 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.759894+0000", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001778151 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762742+0000", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028480049999999998 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762792+0000", 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9521999999999998e-05 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.762831+0000", 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8753e-05 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1058 3.1d 3:bd9a1188:::rbd_data.11cd70c2eac3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.006001532, 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063284730000000003, 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1058 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836422+0000", 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836426+0000", 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836435+0000", 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0610000000000002e-06 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836439+0000", 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4619999999999998e-06 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836483+0000", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-05 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836536+0000", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.837607+0000", 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010716250000000001 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842680+0000", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050721500000000001 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842692+0000", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1960000000000001e-05 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842721+0000", 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9521000000000001e-05 2024-09-16T22:16:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842750+0000", 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8483000000000002e-05 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1247 3.11 3:8841aec0:::rbd_header.11cd70c2eac3:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.052713+0000", 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.789709942, 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047624850000000003, 2024-09-16T22:16:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1247 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052713+0000", 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052713+0000", 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052714+0000", 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.232e-06 2024-09-16T22:16:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052721+0000", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9290000000000001e-06 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052731+0000", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0214000000000001e-05 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052736+0000", 2024-09-16T22:16:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052783+0000", 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6529000000000001e-05 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052846+0000", 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3050000000000001e-05 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053016+0000", 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000170726 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.053493+0000", 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047629999999999998 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.057412+0000", 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039196430000000004 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.057445+0000", 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3114000000000002e-05 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.057475+0000", 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9774e-05 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1345 3.1d 3:bccdc97b:::rbd_data.11cd70c2eac3.0000000000000007:head [stat,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.124122+0000", 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.71830048499999999, 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044626240000000001, 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1345 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124122+0000", 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124122+0000", 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124123+0000", 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.124e-06 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124127+0000", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124150+0000", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2527999999999998e-05 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124158+0000", 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4020000000000002e-06 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124204+0000", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6477999999999998e-05 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124281+0000", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6579e-05 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.124405+0000", 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012421000000000001 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.125687+0000", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012813989999999999 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.128493+0000", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028067510000000001 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.128546+0000", 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3072000000000003e-05 2024-09-16T22:16:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.128585+0000", 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8296000000000001e-05 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1400 3.9 3:923ad63a:::rbd_data.11cd70c2eac3.0000000000000002:head [stat,truncate 1819804] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.177936+0000", 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66448651000000003, 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047033209999999999, 2024-09-16T22:16:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1400 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177936+0000", 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177936+0000", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177939+0000", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9450000000000002e-06 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177954+0000", 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5221e-05 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177985+0000", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1126e-05 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.177997+0000", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1328000000000001e-05 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178125+0000", 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012776899999999999 2024-09-16T22:16:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178203+0000", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8367000000000004e-05 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178339+0000", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013600900000000001 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182081+0000", 2024-09-16T22:16:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003742535 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182537+0000", 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00045572599999999999 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182595+0000", 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8232000000000001e-05 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182639+0000", 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4063000000000001e-05 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1401 3.a 3:57ae193b:::rbd_data.11cd70c2eac3.0000000000000003:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.178048+0000", 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66437435, 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046926839999999999, 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1401 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178048+0000", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178048+0000", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178049+0000", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999996e-07 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178052+0000", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.551e-06 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178060+0000", 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0180000000000001e-06 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178066+0000", 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178699+0000", 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063284300000000001 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178781+0000", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1619000000000006e-05 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.178916+0000", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135193 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182442+0000", 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035260109999999999 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182559+0000", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011755 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182696+0000", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013687700000000001 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.182741+0000", 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4608999999999999e-05 2024-09-16T22:16:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1402 3.14 3:287671ef:::rbd_data.11cd70c2eac3.0000000000000004:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179149+0000", 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66327317200000002, 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059653780000000003, 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1402 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179149+0000", 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179149+0000", 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179150+0000", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8999999999999995e-07 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179154+0000", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8850000000000001e-06 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179163+0000", 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1640000000000007e-06 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179170+0000", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179226+0000", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6042000000000003e-05 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179301+0000", 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5073000000000004e-05 2024-09-16T22:16:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179430+0000", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129443 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.184970+0000", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055393990000000004 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185032+0000", 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1799999999999998e-05 2024-09-16T22:16:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185082+0000", 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0716e-05 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185115+0000", 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2478000000000003e-05 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1403 3.d 3:b26fa55d:::rbd_data.11cd70c2eac3.0000000000000005:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66323430699999997, 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060892699999999999, 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1403 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179189+0000", 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179192+0000", 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.157e-06 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179447+0000", 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025534700000000001 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179457+0000", 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179525+0000", 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599000000000001e-05 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179662+0000", 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013697700000000001 2024-09-16T22:16:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180004+0000", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000341916 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185068+0000", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050640370000000004 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185199+0000", 2024-09-16T22:16:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013083600000000001 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185243+0000", 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4557e-05 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185278+0000", 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4518999999999999e-05 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1405 3.1d 3:bccdc97b:::rbd_data.11cd70c2eac3.0000000000000007:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179608+0000", 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66281431199999996, 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056547059999999998, 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1405 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179608+0000", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179608+0000", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179609+0000", 2024-09-16T22:16:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4000000000000001e-07 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179613+0000", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9430000000000002e-06 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179622+0000", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0899999999999994e-06 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179628+0000", 2024-09-16T22:16:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179680+0000", 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1048000000000003e-05 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179754+0000", 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4481999999999996e-05 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179880+0000", 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012643499999999999 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185060+0000", 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005179365 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185181+0000", 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121086 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185236+0000", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5260000000000003e-05 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185263+0000", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6871000000000001e-05 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1408 3.14 3:2bf0f6b0:::rbd_data.11cd70c2eac3.000000000000000a:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.180111+0000", 2024-09-16T22:16:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66231153600000003, 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050494709999999998, 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1408 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180111+0000", 2024-09-16T22:16:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180111+0000", 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180112+0000", 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180116+0000", 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9179999999999996e-06 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180125+0000", 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7499999999999998e-06 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180132+0000", 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.674e-06 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180275+0000", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000142915 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180354+0000", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9315999999999998e-05 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180490+0000", 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013525000000000001 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185018+0000", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045280659999999999 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185122+0000", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104215 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185144+0000", 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2059999999999999e-05 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185161+0000", 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-05 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1443 3.a 3:56bea48c:::rbd_data.11cd70c2eac3.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.215525+0000", 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62689715700000004, 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044053410000000001, 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1443 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215525+0000", 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215525+0000", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215527+0000", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.44e-06 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215534+0000", 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.289e-06 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215547+0000", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2880999999999999e-05 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215555+0000", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9840000000000001e-06 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215605+0000", 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0214999999999999e-05 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.215659+0000", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3896e-05 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.216397+0000", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073806000000000002 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.219749+0000", 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033517180000000001 2024-09-16T22:16:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.219799+0000", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.969e-05 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.219866+0000", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6977000000000006e-05 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.219931+0000", 2024-09-16T22:16:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5191000000000007e-05 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1563 3.9 3:923ad63a:::rbd_data.11cd70c2eac3.0000000000000002:head [truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.359884+0000", 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.48253913500000001, 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050516659999999998, 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1563 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359884+0000", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359884+0000", 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359885+0000", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.136e-06 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359890+0000", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0900000000000004e-06 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359906+0000", 2024-09-16T22:16:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6401000000000001e-05 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.359914+0000", 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7530000000000001e-06 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360087+0000", 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000172996 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360571+0000", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048440199999999999 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360683+0000", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112138 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.362154+0000", 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001470612 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364841+0000", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002686548 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364885+0000", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4839000000000002e-05 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364935+0000", 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9750999999999999e-05 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1568 3.1d 3:bccdc97b:::rbd_data.11cd70c2eac3.0000000000000007:head [truncate 655360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.360103+0000", 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.48231985199999999, 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047423760000000004, 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1568 2024-09-16T22:16:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360103+0000", 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360103+0000", 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360103+0000", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360105+0000", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2309999999999999e-06 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360112+0000", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-06 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360117+0000", 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360414+0000", 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000296445 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360488+0000", 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3891000000000001e-05 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360607+0000", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119219 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.362452+0000", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00184465 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364748+0000", 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022967759999999999 2024-09-16T22:16:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364804+0000", 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5424999999999999e-05 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364845+0000", 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1356000000000002e-05 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1571 3.14 3:2bf0f6b0:::rbd_data.11cd70c2eac3.000000000000000a:head [truncate 655360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.360124+0000", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.48229820000000001, 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047420240000000001, 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1571 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360124+0000", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360124+0000", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360125+0000", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360127+0000", 2024-09-16T22:16:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0669999999999999e-06 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360133+0000", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1249999999999998e-06 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360137+0000", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4819999999999996e-06 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360528+0000", 2024-09-16T22:16:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039072400000000001 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360597+0000", 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8975999999999998e-05 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.360698+0000", 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100707 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.362820+0000", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002122493 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364782+0000", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019614749999999999 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364829+0000", 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7182000000000003e-05 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.364866+0000", 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7443e-05 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1804 3.a 3:56bea48c:::rbd_data.11cd70c2eac3.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.695267+0000", 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14715586, 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059153720000000003, 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1804 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695267+0000", 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695267+0000", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695269+0000", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0650000000000001e-06 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695279+0000", 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695294+0000", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4956e-05 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695301+0000", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695351+0000", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9567000000000002e-05 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.695399+0000", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8532000000000002e-05 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.696429+0000", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001029613 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.701093+0000", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046640309999999999 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.701121+0000", 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8033000000000001e-05 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.701154+0000", 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3309000000000002e-05 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.701182+0000", 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8059999999999999e-05 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1882 3.9 3:923ad63a:::rbd_data.11cd70c2eac3.0000000000000002:head [truncate 1441792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.047359141, 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006568979, 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1882 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795065+0000", 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.221e-06 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795070+0000", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795085+0000", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4696e-05 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795092+0000", 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7330000000000003e-06 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795165+0000", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2428999999999994e-05 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795259+0000", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4246000000000005e-05 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795379+0000", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011969899999999999 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.796874+0000", 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001495637 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801529+0000", 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046545900000000001 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801587+0000", 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8588999999999998e-05 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801632+0000", 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000002e-05 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:22.958 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:22.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:22.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:395 trunc from 0xd8d7b03 to 0x1863974 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:398 write 0x970bbe9 thru 0x97137df (0x7bf7 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:399 punch from 0xdf0d03 to 0xdf37a8, (0x2aa5 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:401 writesame 0x5837201 thru 0x5845993 data_size 0x15(0xe793 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:407 writesame 0x1dca42c thru 0x1dda032 data_size 0xd(0xfc07 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:408 writesame 0x7a19f79 thru 0x7a253d3 data_size 0x1(0xb45b bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:409 read 0x4f81ed5 thru 0x4f8cd39 (0xae65 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:410 writesame 0xdd44cdd thru 0xdd5389c data_size 0x17(0xebc0 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:411 read 0x8125d8c thru 0x8131dd7 (0xc04c bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:412 writesame 0x536df61 thru 0x5379378 data_size 0x44(0xb418 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:413 read 0x2cf15e1 thru 0x2cf7db7 (0x67d7 bytes) 2024-09-16T22:16:22.962 INFO:teuthology.orchestra.run.smithi031.stdout:415 writesame 0xcba1bfa thru 0xcba2748 data_size 0x5(0xb4f bytes) 2024-09-16T22:16:22.963 INFO:teuthology.orchestra.run.smithi031.stdout:416 trunc from 0xdd5389d to 0x9f3c135 2024-09-16T22:16:22.963 INFO:teuthology.orchestra.run.smithi031.stdout:417 write 0xa589bfe thru 0xa590768 (0x6b6b bytes) 2024-09-16T22:16:22.963 INFO:teuthology.orchestra.run.smithi031.stdout:420 writesame 0x814bf28 thru 0x8152140 data_size 0x3(0x6219 bytes) 2024-09-16T22:16:22.963 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:22.961+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:22.964 INFO:teuthology.orchestra.run.smithi031.stdout:421 read 0x647ebbe thru 0x6482c70 (0x40b3 bytes) 2024-09-16T22:16:22.964 INFO:teuthology.orchestra.run.smithi031.stdout:422 read 0x1606b38 thru 0x160e8a5 (0x7d6e bytes) 2024-09-16T22:16:22.964 INFO:teuthology.orchestra.run.smithi031.stdout:424 read 0x5b09dab thru 0x5b17b75 (0xddcb bytes) 2024-09-16T22:16:22.964 INFO:teuthology.orchestra.run.smithi031.stdout:426 read 0x582799d thru 0x5829188 (0x17ec bytes) 2024-09-16T22:16:22.964 INFO:teuthology.orchestra.run.smithi031.stdout:427 write 0x458571a thru 0x45902e2 (0xabc9 bytes) 2024-09-16T22:16:22.965 INFO:teuthology.orchestra.run.smithi031.stdout:428 punch from 0x11aa90b to 0x11b6900, (0xbff5 bytes) 2024-09-16T22:16:22.965 INFO:teuthology.orchestra.run.smithi031.stdout:429 read 0x4898eaf thru 0x48a08fc (0x7a4e bytes) 2024-09-16T22:16:22.965 INFO:teuthology.orchestra.run.smithi031.stdout:430 read 0x9e00035 thru 0x9e027d6 (0x27a2 bytes) 2024-09-16T22:16:22.965 INFO:teuthology.orchestra.run.smithi031.stdout:432 punch from 0x1aa773b to 0x1aa93bd, (0x1c82 bytes) 2024-09-16T22:16:22.971 INFO:teuthology.orchestra.run.smithi031.stdout:433 writesame 0xc54b6cd thru 0xc552edd data_size 0x1(0x7811 bytes) 2024-09-16T22:16:22.971 INFO:teuthology.orchestra.run.smithi031.stdout:437 write 0x8d46810 thru 0x8d47a22 (0x1213 bytes) 2024-09-16T22:16:22.974 INFO:teuthology.orchestra.run.smithi031.stdout:438 write 0x9d53f72 thru 0x9d5c311 (0x83a0 bytes) 2024-09-16T22:16:22.977 INFO:teuthology.orchestra.run.smithi031.stdout:440 write 0x98cffa1 thru 0x98d032f (0x38f bytes) 2024-09-16T22:16:22.983 INFO:teuthology.orchestra.run.smithi031.stdout:441 read 0xa0e530f thru 0xa0e9918 (0x460a bytes) 2024-09-16T22:16:22.983 INFO:teuthology.orchestra.run.smithi031.stdout:442 write 0x65a66f3 thru 0x65b657a (0xfe88 bytes) 2024-09-16T22:16:22.987 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:22.988 INFO:teuthology.orchestra.run.smithi031.stdout:447 read 0x37ac592 thru 0x37ace35 (0x8a4 bytes) 2024-09-16T22:16:22.988 INFO:teuthology.orchestra.run.smithi031.stdout:448 write 0xbf9a08 thru 0xc08784 (0xed7d bytes) 2024-09-16T22:16:22.990 INFO:teuthology.orchestra.run.smithi031.stdout:449 clone 3 order 24 su 2097152 sc 5 2024-09-16T22:16:23.060 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:23.062+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:23.068 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:23.068 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:23.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:23.089 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:23.089 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:23.106 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:23.106 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:23.122 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:23.140 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:16:23.159 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:23.159 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:23.160 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:23.162+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:23.179 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:23.179 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:23.182 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.182 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:23.198 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:23.215 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:23.216 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:23.276 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:23.278+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:23.326 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T22:16:23.337 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.337 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:23.362 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:23.363+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:23.463 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:23.464+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:23.564 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:23.565+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:23.581 INFO:teuthology.orchestra.run.smithi031.stdout:450 read 0x1c4377c thru 0x1c459c8 (0x224d bytes) 2024-09-16T22:16:23.582 INFO:teuthology.orchestra.run.smithi031.stdout:451 writesame 0x31f5e99 thru 0x31f9cde data_size 0x3(0x3e46 bytes) 2024-09-16T22:16:23.595 INFO:teuthology.orchestra.run.smithi031.stdout:453 read 0x7ae7cf5 thru 0x7aec70a (0x4a16 bytes) 2024-09-16T22:16:23.595 INFO:teuthology.orchestra.run.smithi031.stdout:454 write 0x90b92fd thru 0x90bad98 (0x1a9c bytes) 2024-09-16T22:16:23.601 INFO:teuthology.orchestra.run.smithi031.stdout:456 write 0x85dd1bc thru 0x85eb427 (0xe26c bytes) 2024-09-16T22:16:23.608 INFO:teuthology.orchestra.run.smithi031.stdout:459 read 0x9a65720 thru 0x9a6d1f3 (0x7ad4 bytes) 2024-09-16T22:16:23.609 INFO:teuthology.orchestra.run.smithi031.stdout:461 read 0x4bc6fa7 thru 0x4bd32b2 (0xc30c bytes) 2024-09-16T22:16:23.609 INFO:teuthology.orchestra.run.smithi031.stdout:462 read 0xa89e75c thru 0xa8a33ec (0x4c91 bytes) 2024-09-16T22:16:23.615 INFO:teuthology.orchestra.run.smithi031.stdout:463 trunc from 0xc552ede to 0x6db3d19 2024-09-16T22:16:23.640 INFO:teuthology.orchestra.run.smithi031.stdout:464 write 0x7e7d250 thru 0x7e7ee0d (0x1bbe bytes) 2024-09-16T22:16:23.642 INFO:teuthology.orchestra.run.smithi031.stdout:465 read 0x38dbbf6 thru 0x38e742b (0xb836 bytes) 2024-09-16T22:16:23.643 INFO:teuthology.orchestra.run.smithi031.stdout:466 trunc from 0x7e7ee0e to 0x75bf2ca 2024-09-16T22:16:23.652 INFO:teuthology.orchestra.run.smithi031.stdout:468 trunc from 0x75bf2ca to 0x4b413ab 2024-09-16T22:16:23.653 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.653 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:23.664 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:23.666+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:23.684 INFO:teuthology.orchestra.run.smithi031.stdout:469 write 0xa2b84ed thru 0xa2c7e68 (0xf97c bytes) 2024-09-16T22:16:23.687 INFO:teuthology.orchestra.run.smithi031.stdout:470 punch from 0x7d3d6c3 to 0x7d4606d, (0x89aa bytes) 2024-09-16T22:16:23.687 INFO:teuthology.orchestra.run.smithi031.stdout:471 writesame 0xa2845e thru 0xa370f5 data_size 0x86(0xec98 bytes) 2024-09-16T22:16:23.709 INFO:teuthology.orchestra.run.smithi031.stdout:475 trunc from 0xa2c7e69 to 0x16effe7 2024-09-16T22:16:23.731 INFO:teuthology.orchestra.run.smithi031.stdout:476 trunc from 0x16effe7 to 0x3641381 2024-09-16T22:16:23.752 INFO:teuthology.orchestra.run.smithi031.stdout:477 writesame 0xd11dbe2 thru 0xd12ac07 data_size 0x6b(0xd026 bytes) 2024-09-16T22:16:23.752 INFO:teuthology.orchestra.run.smithi031.stdout:480 write 0x81a02f thru 0x81a9e5 (0x9b7 bytes) 2024-09-16T22:16:23.760 INFO:teuthology.orchestra.run.smithi031.stdout:481 write 0xec58967 thru 0xec5a2d6 (0x1970 bytes) 2024-09-16T22:16:23.765 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:23.766+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:23.765 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:23.768 INFO:teuthology.orchestra.run.smithi031.stdout:483 trunc from 0xec5a2d7 to 0xc97bdc6 2024-09-16T22:16:23.774 INFO:teuthology.orchestra.run.smithi031.stdout:485 read 0xd6653 thru 0xdbc6a (0x5618 bytes) 2024-09-16T22:16:23.774 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.775 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:486 read 0x56947e5 thru 0x5699406 (0x4c22 bytes) 2024-09-16T22:16:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:488 read 0x6f284c9 thru 0x6f351e5 (0xcd1d bytes) 2024-09-16T22:16:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:489 write 0xbfa935 thru 0xc0600f (0xb6db bytes) 2024-09-16T22:16:23.777 INFO:teuthology.orchestra.run.smithi031.stdout:491 write 0x69ef594 thru 0x69f1931 (0x239e bytes) 2024-09-16T22:16:23.780 INFO:teuthology.orchestra.run.smithi031.stdout:493 punch from 0xa34e097 to 0xa3522f9, (0x4262 bytes) 2024-09-16T22:16:23.781 INFO:teuthology.orchestra.run.smithi031.stdout:495 read 0x2a92f55 thru 0x2a9d77c (0xa828 bytes) 2024-09-16T22:16:23.781 INFO:teuthology.orchestra.run.smithi031.stdout:496 write 0x36a23ff thru 0x36af4ce (0xd0d0 bytes) 2024-09-16T22:16:23.782 INFO:teuthology.orchestra.run.smithi031.stdout:497 read 0x1d66ba thru 0x1dfaa1 (0x93e8 bytes) 2024-09-16T22:16:23.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:23.788 INFO:teuthology.orchestra.run.smithi031.stdout:498 write 0xcefb419 thru 0xcf05908 (0xa4f0 bytes) 2024-09-16T22:16:23.790 INFO:teuthology.orchestra.run.smithi031.stdout:499 write 0x5661010 thru 0x5670088 (0xf079 bytes) 2024-09-16T22:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout:500 flatten 2024-09-16T22:16:23.800 INFO:teuthology.orchestra.run.smithi031.stdout:502 punch from 0x744a59b to 0x74592d8, (0xed3d bytes) 2024-09-16T22:16:23.800 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:23.800 INFO:teuthology.orchestra.run.smithi031.stdout:507 read 0x674f23d thru 0x6754456 (0x521a bytes) 2024-09-16T22:16:23.807 INFO:teuthology.orchestra.run.smithi031.stdout:508 write 0xe02a079 thru 0xe0365a7 (0xc52f bytes) 2024-09-16T22:16:23.811 INFO:teuthology.orchestra.run.smithi031.stdout:511 write 0x6594446 thru 0x6597376 (0x2f31 bytes) 2024-09-16T22:16:23.817 INFO:teuthology.orchestra.run.smithi031.stdout:512 writesame 0xd0bf056 thru 0xd0c6718 data_size 0x1(0x76c3 bytes) 2024-09-16T22:16:23.817 INFO:teuthology.orchestra.run.smithi031.stdout:514 read 0xb604fe3 thru 0xb60fe17 (0xae35 bytes) 2024-09-16T22:16:23.817 INFO:teuthology.orchestra.run.smithi031.stdout:515 punch from 0x5a19bfb to 0x5a22bd8, (0x8fdd bytes) 2024-09-16T22:16:23.817 INFO:teuthology.orchestra.run.smithi031.stdout:516 writesame 0x81f4f33 thru 0x81fbe6d data_size 0x19(0x6f3b bytes) 2024-09-16T22:16:23.820 INFO:teuthology.orchestra.run.smithi031.stdout:517 trunc from 0xe0365a8 to 0xe688b6f 2024-09-16T22:16:23.837 INFO:teuthology.orchestra.run.smithi031.stdout:518 trunc from 0xe688b6f to 0x407daec 2024-09-16T22:16:23.857 INFO:teuthology.orchestra.run.smithi031.stdout:519 write 0x178c066 thru 0x1793dac (0x7d47 bytes) 2024-09-16T22:16:23.866 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:23.867+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:23.872 INFO:teuthology.orchestra.run.smithi031.stdout:523 write 0x8f66cd6 thru 0x8f6b664 (0x498f bytes) 2024-09-16T22:16:23.875 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:23.888 INFO:teuthology.orchestra.run.smithi031.stdout:526 write 0xec3b097 thru 0xec4725f (0xc1c9 bytes) 2024-09-16T22:16:23.891 INFO:teuthology.orchestra.run.smithi031.stdout:528 read 0x2350011 thru 0x23553e2 (0x53d2 bytes) 2024-09-16T22:16:23.892 INFO:teuthology.orchestra.run.smithi031.stdout:531 punch from 0xe8ab7b1 to 0xe8b74b5, (0xbd04 bytes) 2024-09-16T22:16:23.893 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:23.893 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:23.895 INFO:teuthology.orchestra.run.smithi031.stdout:532 trunc from 0xec47260 to 0xbde2ae0 2024-09-16T22:16:23.902 INFO:teuthology.orchestra.run.smithi031.stdout:533 read 0xa9c3a94 thru 0xa9c4b74 (0x10e1 bytes) 2024-09-16T22:16:23.902 INFO:teuthology.orchestra.run.smithi031.stdout:536 read 0xb826774 thru 0xb830d04 (0xa591 bytes) 2024-09-16T22:16:23.902 INFO:teuthology.orchestra.run.smithi031.stdout:537 read 0x7404b87 thru 0x741070b (0xbb85 bytes) 2024-09-16T22:16:23.902 INFO:teuthology.orchestra.run.smithi031.stdout:538 write 0x4ed4e65 thru 0x4ed694a (0x1ae6 bytes) 2024-09-16T22:16:23.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:23.903 INFO:teuthology.orchestra.run.smithi031.stdout:539 writesame 0x9b3a2c5 thru 0x9b48613 data_size 0x33(0xe34f bytes) 2024-09-16T22:16:23.920 INFO:teuthology.orchestra.run.smithi031.stdout:542 trunc from 0xbde2ae0 to 0x3b2e39 2024-09-16T22:16:23.939 INFO:teuthology.orchestra.run.smithi031.stdout:544 punch from 0x2c04e5 to 0x2c2b9a, (0x26b5 bytes) 2024-09-16T22:16:23.939 INFO:teuthology.orchestra.run.smithi031.stdout:547 read 0x3aa74c thru 0x3adfe8 (0x389d bytes) 2024-09-16T22:16:23.948 INFO:teuthology.orchestra.run.smithi031.stdout:548 trunc from 0x3b2e39 to 0x803e62e 2024-09-16T22:16:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:549 writesame 0x16fe164 thru 0x170cd1a data_size 0x1(0xebb7 bytes) 2024-09-16T22:16:23.967 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:23.968+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:23.983 INFO:teuthology.orchestra.run.smithi031.stdout:550 read 0x1f29132 thru 0x1f29966 (0x835 bytes) 2024-09-16T22:16:23.983 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:23.983 INFO:teuthology.orchestra.run.smithi031.stdout:552 read 0x5c53f3d thru 0x5c5d83b (0x98ff bytes) 2024-09-16T22:16:23.983 INFO:teuthology.orchestra.run.smithi031.stdout:554 clone 4 order 18 su 4096 sc 9 2024-09-16T22:16:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:24.104 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:24.106+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:24.168 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:24.170+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:24.169 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.169 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:24.261 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T22:16:24.269 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:24.271+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:24.322 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.322 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:24.374 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:24.376+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:24.433 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.434 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:24.474 INFO:teuthology.orchestra.run.smithi031.stdout:555 punch from 0x3b97af6 to 0x3b9df6c, (0x6476 bytes) 2024-09-16T22:16:24.481 INFO:teuthology.orchestra.run.smithi031.stdout:557 clone 5 order 22 su 524288 sc 14 2024-09-16T22:16:24.483 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:24.485+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:24.572 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:24.574+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:24.586 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.586 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:24.673 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:24.675+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:24.703 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.703 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:24.795 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:24.797+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:24.858 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:24.858 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:24.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:24.908 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:24.910+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:24.977 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:24.978+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:25.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:25.077 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:25.079+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:25.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:16:25.174 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:25.174 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:25.178 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:25.180+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:25.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:25.221 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone4 from 0x803e62e (overlap 0x803e62e) to 0x0 2024-09-16T22:16:25.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:25.277 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:25.277 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:25.277 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.9312608517144063 2024-09-16T22:16:25.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.9312608517144063 2024-09-16T22:16:25.279 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:25.281+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:25.381 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:25.383+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:25.385 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:25.385 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:25.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:25.466 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:25.482 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:25.484+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:25.511 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:25.552 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:25.552 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.190774060999999, 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015694280000000001, 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391487+0000", 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391488+0000", 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6279999999999999e-06 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391496+0000", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9119999999999998e-06 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391507+0000", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0251000000000001e-05 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391515+0000", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1890000000000007e-06 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391562+0000", 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7051000000000001e-05 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.391570+0000", 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1899999999999995e-06 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.401974+0000", 2024-09-16T22:16:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010404096999999999 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402150+0000", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017572 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402318+0000", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016847499999999999 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.402960+0000", 2024-09-16T22:16:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064145900000000004 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407117+0000", 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041574799999999999 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407153+0000", 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5587000000000003e-05 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.407181+0000", 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8240999999999999e-05 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.052133941000001, 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077487449999999996, 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530127+0000", 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530130+0000", 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530135+0000", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6659999999999996e-06 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530146+0000", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0379e-05 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531751+0000", 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001605022 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531840+0000", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9159000000000005e-05 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.531986+0000", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000145978 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534511+0000", 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025251330000000002 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537823+0000", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033120950000000001 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537855+0000", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1955999999999997e-05 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537876+0000", 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0789999999999999e-05 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051969021000001, 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007677835, 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 43 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530292+0000", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530299+0000", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530304+0000", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530307+0000", 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.095e-06 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532608+0000", 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023004319999999998 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532653+0000", 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5506000000000001e-05 2024-09-16T22:16:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.532734+0000", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0937000000000001e-05 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534517+0000", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001782596 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537905+0000", 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033881570000000001 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537945+0000", 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0624000000000003e-05 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537970+0000", 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4216999999999999e-05 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051890363999998, 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076823799999999999, 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 44 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530371+0000", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530444+0000", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2640999999999998e-05 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530451+0000", 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530455+0000", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0690000000000003e-06 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533032+0000", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025771930000000002 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533083+0000", 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1218999999999999e-05 2024-09-16T22:16:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533162+0000", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8898000000000003e-05 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535469+0000", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023075109999999999 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538005+0000", 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002535388 2024-09-16T22:16:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538032+0000", 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6954e-05 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538053+0000", 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1080000000000001e-05 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051799502000001, 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081949469999999993, 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530461+0000", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530462+0000", 2024-09-16T22:16:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530464+0000", 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530468+0000", 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5410000000000001e-06 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530471+0000", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533574+0000", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031031169999999999 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533631+0000", 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6786000000000001e-05 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.533707+0000", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6099000000000004e-05 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535480+0000", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017729169999999999 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538616+0000", 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031358829999999999 2024-09-16T22:16:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538639+0000", 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3315e-05 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538656+0000", 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7084000000000001e-05 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051785554999999, 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082594290000000004, 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 46 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530475+0000", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530476+0000", 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6899999999999999e-07 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530478+0000", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530481+0000", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3689999999999998e-06 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530483+0000", 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534007+0000", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035237409999999999 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534089+0000", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534239+0000", 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149615 2024-09-16T22:16:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536005+0000", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017664099999999999 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538687+0000", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026822949999999999 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538714+0000", 2024-09-16T22:16:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7236000000000001e-05 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538735+0000", 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0293e-05 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051747012, 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082918049999999993, 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 47 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530514+0000", 2024-09-16T22:16:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530608+0000", 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4163e-05 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530614+0000", 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530617+0000", 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534562+0000", 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039451900000000003 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534617+0000", 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5281999999999997e-05 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.534695+0000", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7344999999999999e-05 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536013+0000", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013184970000000001 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538757+0000", 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027440049999999999 2024-09-16T22:16:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538787+0000", 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9734e-05 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.538806+0000", 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8678e-05 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051637450000001, 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010151530000000001, 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 48 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530623+0000", 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530624+0000", 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4400000000000001e-07 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530626+0000", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530629+0000", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3519999999999998e-06 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530632+0000", 2024-09-16T22:16:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535102+0000", 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044695769999999998 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535147+0000", 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5874999999999997e-05 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535225+0000", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7262000000000007e-05 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536945+0000", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001720297 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540728+0000", 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003783472 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540754+0000", 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5791000000000001e-05 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540775+0000", 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0733999999999998e-05 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051604561000001, 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010193045, 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 49 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530656+0000", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530657+0000", 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-16T22:16:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530660+0000", 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4180000000000001e-06 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530666+0000", 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7370000000000003e-06 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530674+0000", 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0399999999999993e-06 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535615+0000", 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049413870000000002 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535666+0000", 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.092e-05 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.535755+0000", 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8263999999999994e-05 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536950+0000", 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001195776 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540802+0000", 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038521749999999998 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540828+0000", 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5731000000000001e-05 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.540849+0000", 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1182000000000001e-05 2024-09-16T22:16:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051522216999999, 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010330358, 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530739+0000", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530828+0000", 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8504999999999994e-05 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530835+0000", 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7179999999999996e-06 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530839+0000", 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536044+0000", 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052045609999999999 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536092+0000", 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100999999999999e-05 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536171+0000", 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9357999999999995e-05 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536954+0000", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078211999999999997 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541028+0000", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040741529999999996 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541052+0000", 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4267e-05 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541069+0000", 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7088999999999999e-05 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051414170000001, 2024-09-16T22:16:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010291738999999999, 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 51 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530847+0000", 2024-09-16T22:16:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.39e-07 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530849+0000", 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2639999999999998e-06 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530853+0000", 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5279999999999999e-06 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530856+0000", 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536566+0000", 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057098060000000004 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536617+0000", 2024-09-16T22:16:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0711000000000003e-05 2024-09-16T22:16:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536691+0000", 2024-09-16T22:16:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4536999999999995e-05 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537339+0000", 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064814099999999997 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541092+0000", 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037528790000000002 2024-09-16T22:16:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541117+0000", 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5279999999999999e-05 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541138+0000", 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1047999999999999e-05 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.051400753999999, 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010348868000000001, 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 52 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530860+0000", 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530863+0000", 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530867+0000", 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.530879+0000", 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.536976+0000", 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060963989999999997 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537037+0000", 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1623000000000003e-05 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537121+0000", 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3033000000000003e-05 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.537655+0000", 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053447399999999995 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541165+0000", 2024-09-16T22:16:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035098120000000002 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541190+0000", 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5313e-05 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.541209+0000", 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8878000000000001e-05 2024-09-16T22:16:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737381879000001, 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089954470000000002, 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 70 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844879+0000", 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0199999999999998e-07 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844882+0000", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844886+0000", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844889+0000", 2024-09-16T22:16:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848458+0000", 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035685589999999998 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848544+0000", 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6298000000000006e-05 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.848675+0000", 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000130808 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851222+0000", 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025471500000000002 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853828+0000", 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026053309999999998 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853852+0000", 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.495e-05 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853874+0000", 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1982999999999999e-05 2024-09-16T22:16:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737367497000001, 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090279209999999995, 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 71 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844893+0000", 2024-09-16T22:16:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844894+0000", 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844896+0000", 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.069e-06 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844899+0000", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844902+0000", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849147+0000", 2024-09-16T22:16:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042455749999999997 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849225+0000", 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7844999999999998e-05 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849353+0000", 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012841400000000001 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851381+0000", 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020275390000000001 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853879+0000", 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024981949999999999 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853900+0000", 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1138e-05 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853921+0000", 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1029000000000001e-05 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737296951000001, 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011701283, 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 72 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.844964+0000", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-07 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845021+0000", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6480000000000001e-05 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845026+0000", 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3449999999999996e-06 2024-09-16T22:16:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845033+0000", 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.849957+0000", 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049242820000000003 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850000+0000", 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3381999999999999e-05 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.850076+0000", 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853789+0000", 2024-09-16T22:16:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00371274 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856571+0000", 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002782432 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856623+0000", 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1907000000000003e-05 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856665+0000", 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2347e-05 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737221622, 2024-09-16T22:16:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011700838, 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 73 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845039+0000", 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845040+0000", 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845042+0000", 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845047+0000", 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0860000000000001e-06 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845057+0000", 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7149999999999993e-06 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851416+0000", 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063595580000000004 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851460+0000", 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510999999999997e-05 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851531+0000", 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1266000000000005e-05 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853820+0000", 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022893879999999998 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856673+0000", 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028527359999999998 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856720+0000", 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7027000000000002e-05 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856740+0000", 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9973999999999999e-05 2024-09-16T22:16:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737196037, 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011717971000000001, 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 74 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845065+0000", 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-16T22:16:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845070+0000", 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000004e-06 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845074+0000", 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9519999999999996e-06 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845076+0000", 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000001e-06 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851800+0000", 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067234740000000001 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.851883+0000", 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2935999999999993e-05 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852017+0000", 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013409199999999999 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853945+0000", 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019281179999999999 2024-09-16T22:16:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856745+0000", 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002800276 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856766+0000", 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0370999999999999e-05 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.856783+0000", 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7132000000000001e-05 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737179974, 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014248581999999999, 2024-09-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 75 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845081+0000", 2024-09-16T22:16:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845140+0000", 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9061000000000003e-05 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845146+0000", 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845150+0000", 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5899999999999999e-06 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852357+0000", 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072068239999999997 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852403+0000", 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5924999999999998e-05 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.852502+0000", 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9129000000000006e-05 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854543+0000", 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020415730000000001 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859274+0000", 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047301799999999996 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859305+0000", 2024-09-16T22:16:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1117e-05 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859329+0000", 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4760999999999999e-05 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.737102136000001, 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014219621, 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 76 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845159+0000", 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845165+0000", 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999998e-06 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845169+0000", 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9589999999999997e-06 2024-09-16T22:16:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.845172+0000", 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.709e-06 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.853983+0000", 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088106799999999996 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854065+0000", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2452000000000003e-05 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.854200+0000", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013444600000000001 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.855726+0000", 2024-09-16T22:16:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015263049999999999 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859334+0000", 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036082890000000002 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859355+0000", 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0820999999999999e-05 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:58.859378+0000", 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3068e-05 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4201.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.452881871999999, 2024-09-16T22:16:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071099739999999998, 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4201", 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.92:0/875207962", 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 94 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129379+0000", 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129380+0000", 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129383+0000", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8850000000000001e-06 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129390+0000", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.129396+0000", 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131718+0000", 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023225619999999998 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131759+0000", 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0601e-05 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.131829+0000", 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9830000000000001e-05 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.135241+0000", 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003412567 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136401+0000", 2024-09-16T22:16:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011601630000000001 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136452+0000", 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1113999999999999e-05 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:15:59.136489+0000", 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6575999999999997e-05 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:25.715 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:25.584+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:25.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:25.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:25.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:25.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:25.720 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:25.721 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:25.721+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:25.820 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:25.822+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:10995 3.16 3:6b75e1bc:::rbd_data.11cdc482f8dc.00000000000000ea:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.918091+0000", 2024-09-16T22:16:25.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0050611679999999996, 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050872060000000004, 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10995 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918091+0000", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918091+0000", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918092+0000", 2024-09-16T22:16:25.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6109999999999999e-06 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918097+0000", 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6110000000000004e-06 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918112+0000", 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5387e-05 2024-09-16T22:16:25.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918119+0000", 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4269999999999996e-06 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918171+0000", 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2386999999999998e-05 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.918278+0000", 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106564 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:11000 3.3 e31/23 3:c351fba4:::rbd_data.11cdc482f8dc.00000000000000e8:head v 31'45, mlcod=31'43)", 2024-09-16T22:16:25.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.921312+0000", 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0018399200000000001, 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019332990000000001, 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921312+0000", 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921312+0000", 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921313+0000", 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4600000000000004e-07 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921319+0000", 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921330+0000", 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1477e-05 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921337+0000", 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4989999999999999e-06 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921390+0000", 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2621000000000003e-05 2024-09-16T22:16:25.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.921409+0000", 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9012999999999999e-05 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 2 2024-09-16T22:16:25.931 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:25.932 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:25.923+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:25.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:26.022 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:26.023+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:26.123 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:26.125+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:26.126 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:26.126 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:26.126 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:26.127 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:26.127 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:26.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:16:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 0.931261 (ee67) 2024-09-16T22:16:26.224 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:26.226+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:26.314 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1095 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [stat,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4345214869999996, 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063311330000000001, 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1095 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881048+0000", 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881049+0000", 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5680000000000001e-06 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881056+0000", 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3430000000000001e-06 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881071+0000", 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4824000000000001e-05 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881079+0000", 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2400000000000007e-06 2024-09-16T22:16:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881145+0000", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6544999999999994e-05 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881231+0000", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6015999999999998e-05 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881363+0000", 2024-09-16T22:16:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013146499999999999 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.882137+0000", 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077436300000000003 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887295+0000", 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051576399999999998 2024-09-16T22:16:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887330+0000", 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5369e-05 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887379+0000", 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8760000000000001e-05 2024-09-16T22:16:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1101 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4339702479999996, 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005882254, 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1101 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881599+0000", 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881600+0000", 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000004e-07 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881604+0000", 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8079999999999998e-06 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881614+0000", 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0451e-05 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881621+0000", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0829999999999998e-06 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881671+0000", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9738999999999999e-05 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881747+0000", 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5634000000000001e-05 2024-09-16T22:16:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.881868+0000", 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121123 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.884524+0000", 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026559230000000001 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887415+0000", 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028909529999999999 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887443+0000", 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8844e-05 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.887481+0000", 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7926000000000002e-05 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1240 3.e 3:77e2e4fa:::rbd_data.11cd70c2eac3.0000000000000017:head [stat,truncate 4063232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2697232090000004, 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00646095, 2024-09-16T22:16:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1240 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045846+0000", 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045847+0000", 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1170000000000001e-06 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045853+0000", 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045870+0000", 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.715e-05 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045879+0000", 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7350000000000002e-06 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.045950+0000", 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1886999999999998e-05 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046034+0000", 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-05 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046175+0000", 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014056600000000001 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.047145+0000", 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00097021599999999998 2024-09-16T22:16:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052211+0000", 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050656750000000004 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052262+0000", 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660999999999999e-05 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052307+0000", 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4718000000000001e-05 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1246 3.e 3:70842919:::rbd_data.11cd70c2eac3.000000000000001d:head [stat,truncate 4063232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2691141769999996, 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006004995, 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1246 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046455+0000", 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046456+0000", 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1999999999999998e-07 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046460+0000", 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0350000000000003e-06 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046472+0000", 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2167000000000001e-05 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046479+0000", 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046528+0000", 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8717999999999997e-05 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046606+0000", 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8794999999999994e-05 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.046748+0000", 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014167699999999999 2024-09-16T22:16:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.049628+0000", 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028798019999999999 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052360+0000", 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027316020000000001 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052417+0000", 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7003999999999998e-05 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.052460+0000", 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3572000000000002e-05 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1929 3.e 3:73158de0:::rbd_data.11cd70c2eac3.0000000000000009:head [truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.865034+0000", 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.4505356489999999, 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058974639999999998, 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1929 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865034+0000", 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865034+0000", 2024-09-16T22:16:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865036+0000", 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8789999999999999e-06 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865042+0000", 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0950000000000001e-06 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865061+0000", 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9809e-05 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865070+0000", 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4789999999999996e-06 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865137+0000", 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7442999999999998e-05 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865214+0000", 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6647e-05 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865328+0000", 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011363500000000001 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.867290+0000", 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019619300000000002 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870837+0000", 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035476969999999998 2024-09-16T22:16:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870887+0000", 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0049000000000001e-05 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870931+0000", 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3801000000000003e-05 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1932 3.16 3:6b0dcb7f:::rbd_data.11cd70c2eac3.000000000000000c:head [truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.865122+0000", 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.4504472960000001, 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00552068, 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1932 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865122+0000", 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865122+0000", 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865123+0000", 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0800000000000004e-07 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865127+0000", 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8199999999999998e-06 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865135+0000", 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8319999999999995e-06 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865141+0000", 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9460000000000003e-06 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865194+0000", 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2850999999999999e-05 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865260+0000", 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6276e-05 2024-09-16T22:16:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.865362+0000", 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010155299999999999 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.867316+0000", 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019541900000000002 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870545+0000", 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032288450000000002 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870601+0000", 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6248000000000001e-05 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.870643+0000", 2024-09-16T22:16:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1411e-05 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:9869 3.1e e31/23 3:7faa808b:::rbd_data.11cdc482f8dc.00000000000000b2:head v 31'19, mlcod=31'17)", 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.804334+0000", 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.51123507300000004, 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060266909999999998, 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804334+0000", 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804334+0000", 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804335+0000", 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804342+0000", 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9240000000000001e-06 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804354+0000", 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2312e-05 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804361+0000", 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804420+0000", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8928000000000002e-05 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.804438+0000", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8023999999999999e-05 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.810309+0000", 2024-09-16T22:16:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058714539999999999 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.810361+0000", 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1681000000000002e-05 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:9872 3.11 e31/23 3:89d9ffed:::rbd_data.11cdc482f8dc.00000000000000b3:head v 31'116, mlcod=31'114)", 2024-09-16T22:16:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.808163+0000", 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.50740674100000005, 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074948719999999996, 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808163+0000", 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808163+0000", 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808163+0000", 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-07 2024-09-16T22:16:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808169+0000", 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8690000000000001e-06 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808183+0000", 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3847e-05 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808190+0000", 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808243+0000", 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3464999999999997e-05 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808262+0000", 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8921000000000001e-05 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815586+0000", 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073241160000000003 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815657+0000", 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1334999999999994e-05 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.11 epoch 31/23 rep_tid 454 entries 31'119 (0'0) error 3:8a7471e7:::rbd_data.11cdc482f8dc.0000000000000132:head by client.4557.0:13070 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'117)", 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.091141+0000", 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.224427975, 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064499819999999999, 2024-09-16T22:16:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091141+0000", 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091141+0000", 2024-09-16T22:16:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091142+0000", 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3300000000000001e-07 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091151+0000", 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3899999999999993e-06 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091171+0000", 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9976999999999999e-05 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091186+0000", 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5379999999999998e-05 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.091274+0000", 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7916000000000003e-05 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.097591+0000", 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063174859999999998 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13232 3.5 3:a2ec2e80:::rbd_data.11cdc482f8dc.0000000000000149:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.111584+0000", 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.20398524700000001, 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056477209999999996, 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13232 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111584+0000", 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111584+0000", 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111585+0000", 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2580000000000001e-06 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111592+0000", 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9940000000000003e-06 2024-09-16T22:16:26.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111604+0000", 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1780000000000001e-05 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111611+0000", 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1659999999999997e-06 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111668+0000", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7074000000000003e-05 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.111767+0000", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8539e-05 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.117232+0000", 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054649099999999999 2024-09-16T22:16:26.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13385 3.e 3:7241f9c0:::rbd_data.11cdc482f8dc.000000000000018f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.174832+0000", 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14073780999999999, 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056590379999999999, 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13385 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174832+0000", 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174832+0000", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174833+0000", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0109999999999999e-06 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174838+0000", 2024-09-16T22:16:26.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174853+0000", 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4467000000000001e-05 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174860+0000", 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.883e-06 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174916+0000", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6665e-05 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175007+0000", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0518000000000003e-05 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175197+0000", 2024-09-16T22:16:26.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189886 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177503+0000", 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023059899999999999 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180387+0000", 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002883998 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180449+0000", 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2456e-05 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180491+0000", 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1319999999999997e-05 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13386 3.e 3:713ae2d3:::rbd_data.11cdc482f8dc.0000000000000192:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.174972+0000", 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14059777600000001, 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081459549999999999, 2024-09-16T22:16:26.386 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13386 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174972+0000", 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174972+0000", 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174973+0000", 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-06 2024-09-16T22:16:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174979+0000", 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6490000000000002e-06 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174991+0000", 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1477999999999999e-05 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.174998+0000", 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6799999999999993e-06 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175510+0000", 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051185300000000005 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175584+0000", 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3930999999999994e-05 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175766+0000", 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000181814 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180531+0000", 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047654079999999996 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183009+0000", 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024772449999999999 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183073+0000", 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4033000000000002e-05 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183118+0000", 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4814999999999997e-05 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:13387 3.1e e31/23 3:7b9a15c0:::rbd_data.11cdc482f8dc.0000000000000190:head v 31'35, mlcod=31'33)", 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.175450+0000", 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.140118878, 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075487219999999999, 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175450+0000", 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175450+0000", 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175451+0000", 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5000000000000001e-07 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175457+0000", 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.57e-06 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175468+0000", 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1104000000000001e-05 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175480+0000", 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1601e-05 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175564+0000", 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4136999999999998e-05 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.175583+0000", 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9656999999999998e-05 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182956+0000", 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073727519999999998 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182999+0000", 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3050999999999998e-05 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.1 epoch 31/23 rep_tid 125 entries 31'34 (0'0) error 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head by client.4557.0:13391 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'32)", 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.176954+0000", 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.138615351, 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006096143, 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176954+0000", 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176954+0000", 2024-09-16T22:16:26.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176955+0000", 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176962+0000", 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-16T22:16:26.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176978+0000", 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5443000000000001e-05 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176995+0000", 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7275999999999999e-05 2024-09-16T22:16:26.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177062+0000", 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6993e-05 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183050+0000", 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059882499999999996 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.11 epoch 31/23 rep_tid 499 entries 31'129 (0'0) error 3:89f76da2:::rbd_data.11cdc482f8dc.0000000000000195:head by client.4557.0:13389 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'127)", 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.177059+0000", 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.13851079099999999, 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061696829999999996, 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177059+0000", 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177059+0000", 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177059+0000", 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177064+0000", 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177081+0000", 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.683e-05 2024-09-16T22:16:26.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177094+0000", 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2818e-05 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.177166+0000", 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2390000000000003e-05 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183228+0000", 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060617450000000003 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13394 3.e 3:74c61277:::rbd_data.11cdc482f8dc.0000000000000199:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.180574+0000", 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.13499517899999999, 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014149831999999999, 2024-09-16T22:16:26.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13394 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180574+0000", 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180574+0000", 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180575+0000", 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3799999999999996e-07 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180579+0000", 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7969999999999998e-06 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180588+0000", 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6700000000000006e-06 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180596+0000", 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1149999999999996e-06 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180635+0000", 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9816e-05 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180640+0000", 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0869999999999997e-06 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188215+0000", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075741530000000001 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188351+0000", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013612800000000001 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194724+0000", 2024-09-16T22:16:26.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063732279999999999 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:13403 3.14 e32/23 3:2f33ca34:::rbd_data.11cdc482f8dc.0000000000000198:head v 32'54, mlcod=31'52)", 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.186276+0000", 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.129293398, 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055836899999999997, 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186276+0000", 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186276+0000", 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186277+0000", 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4000000000000001e-07 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186282+0000", 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1719999999999998e-06 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186291+0000", 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4409999999999999e-06 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186297+0000", 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1020000000000002e-06 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186374+0000", 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6259000000000003e-05 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186392+0000", 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7978e-05 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191804+0000", 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054128520000000001 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191860+0000", 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5245999999999999e-05 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13444 3.5 3:a07194e2:::rbd_data.11cdc482f8dc.00000000000001b1:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.208339+0000", 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.107230719, 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060582090000000002, 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13444 2024-09-16T22:16:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208339+0000", 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208339+0000", 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208341+0000", 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.605e-06 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208349+0000", 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4499999999999998e-06 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208367+0000", 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8601999999999999e-05 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208377+0000", 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9820000000000003e-06 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208445+0000", 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7597999999999999e-05 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.208576+0000", 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131268 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.214397+0000", 2024-09-16T22:16:26.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058207040000000003 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.3 epoch 32/23 rep_tid 387 entries 32'62 (0'0) error 3:c52f59ad:::rbd_data.11cdc482f8dc.00000000000001ed:head by client.4557.0:13565 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'60)", 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.257873+0000", 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.057696211999999997, 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055175140000000003, 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257873+0000", 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257873+0000", 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257874+0000", 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257883+0000", 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5949999999999999e-06 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257903+0000", 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9905e-05 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.257936+0000", 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2812999999999999e-05 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.258001+0000", 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5777000000000004e-05 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.263391+0000", 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053892640000000004 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13595 3.5 3:a7f0f76d:::rbd_data.11cdc482f8dc.00000000000001fd:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.270196+0000", 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045372858000000002, 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058888780000000002, 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13595 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270196+0000", 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270196+0000", 2024-09-16T22:16:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270197+0000", 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000001e-07 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270201+0000", 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.07e-06 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270211+0000", 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8279999999999998e-06 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270216+0000", 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270264+0000", 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8139000000000001e-05 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270316+0000", 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1700999999999998e-05 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270422+0000", 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106728 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.270882+0000", 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000459734 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.276011+0000", 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051284520000000004 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.276048+0000", 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6909000000000002e-05 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.276085+0000", 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7789e-05 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.435 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:26.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:26.438 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:26.438+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:26.537 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:26.539+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:26.545 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:26.545 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:26.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:26.638 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:26.640+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:26.660 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:26.661 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:26.661 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:26.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:26.739 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:26.741+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:26.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.350935996, 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:26.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:26.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:26.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1058 3.1d 3:bd9a1188:::rbd_data.11cd70c2eac3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9420051110000003, 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063284730000000003, 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1058 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836422+0000", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836426+0000", 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836435+0000", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0610000000000002e-06 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836439+0000", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4619999999999998e-06 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836483+0000", 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-05 2024-09-16T22:16:26.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836536+0000", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.837607+0000", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010716250000000001 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842680+0000", 2024-09-16T22:16:26.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050721500000000001 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842692+0000", 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1960000000000001e-05 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842721+0000", 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9521000000000001e-05 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842750+0000", 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8483000000000002e-05 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1403 3.d 3:b26fa55d:::rbd_data.11cd70c2eac3.0000000000000005:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5992378860000001, 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060892699999999999, 2024-09-16T22:16:26.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1403 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:26.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179189+0000", 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179192+0000", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.157e-06 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179447+0000", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025534700000000001 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179457+0000", 2024-09-16T22:16:26.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179525+0000", 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599000000000001e-05 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179662+0000", 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013697700000000001 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180004+0000", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000341916 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185068+0000", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050640370000000004 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185199+0000", 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013083600000000001 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185243+0000", 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4557e-05 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185278+0000", 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4518999999999999e-05 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1882 3.9 3:923ad63a:::rbd_data.11cd70c2eac3.0000000000000002:head [truncate 1441792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.9833627200000001, 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006568979, 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1882 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795065+0000", 2024-09-16T22:16:26.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.221e-06 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795070+0000", 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795085+0000", 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4696e-05 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795092+0000", 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7330000000000003e-06 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795165+0000", 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2428999999999994e-05 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795259+0000", 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4246000000000005e-05 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795379+0000", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011969899999999999 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.796874+0000", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001495637 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801529+0000", 2024-09-16T22:16:26.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046545900000000001 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801587+0000", 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8588999999999998e-05 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801632+0000", 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000002e-05 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1963 3.14 3:281b8854:::rbd_data.11cd70c2eac3.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8754130419999999, 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01015051, 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1963 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903014+0000", 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903018+0000", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0010000000000003e-06 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903028+0000", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903034+0000", 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4670000000000001e-06 2024-09-16T22:16:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903076+0000", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1900000000000002e-05 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903123+0000", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7178000000000001e-05 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.904508+0000", 2024-09-16T22:16:26.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013844110000000001 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913089+0000", 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085809930000000003 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913100+0000", 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0868e-05 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913132+0000", 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2632999999999997e-05 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913164+0000", 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1316000000000003e-05 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9872 3.11 3:89d9ffed:::rbd_data.11cdc482f8dc.00000000000000b3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.97070008799999996, 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082067210000000002, 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9872 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807727+0000", 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5000000000000001e-07 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807731+0000", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807740+0000", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9320000000000002e-06 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807746+0000", 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1720000000000004e-06 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807804+0000", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8088999999999999e-05 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807873+0000", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8621999999999995e-05 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808064+0000", 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019087999999999999 2024-09-16T22:16:26.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.810113+0000", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020487769999999999 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815829+0000", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057160559999999997 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815885+0000", 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489000000000001e-05 2024-09-16T22:16:26.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815933+0000", 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7728000000000001e-05 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9883 3.d 3:b5c08af5:::rbd_data.11cdc482f8dc.00000000000000bc:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.96482736499999999, 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063131180000000004, 2024-09-16T22:16:26.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9883 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813600+0000", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6899999999999997e-07 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813602+0000", 2024-09-16T22:16:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9289999999999998e-06 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813878+0000", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027559200000000001 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813888+0000", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5349999999999993e-06 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.814099+0000", 2024-09-16T22:16:26.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000211177 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.814223+0000", 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012391100000000001 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.819912+0000", 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056893050000000004 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9899 3.d 3:b5c08af5:::rbd_data.11cdc482f8dc.00000000000000bc:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.95762792399999996, 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066083340000000004, 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9899 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820799+0000", 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4600000000000003e-07 2024-09-16T22:16:26.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820805+0000", 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6359999999999999e-06 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820817+0000", 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2554e-05 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820824+0000", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821023+0000", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019892600000000001 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821102+0000", 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8920999999999999e-05 2024-09-16T22:16:26.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821307+0000", 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020460899999999999 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.822279+0000", 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00097247000000000002 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827317+0000", 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050373709999999997 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827372+0000", 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4978000000000001e-05 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827407+0000", 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5136999999999999e-05 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9907 3.a 3:55fa9e5b:::rbd_data.11cdc482f8dc.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.95347847200000002, 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086142360000000008, 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9907 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824949+0000", 2024-09-16T22:16:26.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0400000000000005e-07 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824953+0000", 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.873e-06 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824963+0000", 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0061000000000001e-05 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824967+0000", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825021+0000", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067999999999998e-05 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825109+0000", 2024-09-16T22:16:26.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7311000000000005e-05 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825307+0000", 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000198481 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.829904+0000", 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045966729999999999 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833459+0000", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035554599999999999 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833521+0000", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1753000000000003e-05 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833562+0000", 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1017999999999999e-05 2024-09-16T22:16:26.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13070 3.11 3:8a7471e7:::rbd_data.11cdc482f8dc.0000000000000132:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68797732300000003, 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073813500000000001, 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13070 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090451+0000", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.922e-06 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090458+0000", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6290000000000001e-06 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090484+0000", 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6038e-05 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090497+0000", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2371999999999999e-05 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090565+0000", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7861999999999995e-05 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090690+0000", 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012561399999999999 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.097830+0000", 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071399130000000003 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13330 3.13 3:cb153b9d:::rbd_data.11cdc482f8dc.0000000000000172:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.63307366099999995, 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091642049999999999, 2024-09-16T22:16:26.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13330 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145354+0000", 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5900000000000005e-07 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145359+0000", 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145370+0000", 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1107000000000001e-05 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145375+0000", 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145428+0000", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3375000000000003e-05 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145511+0000", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2578999999999997e-05 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145703+0000", 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019194200000000001 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150012+0000", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043096810000000001 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154427+0000", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004414935 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154483+0000", 2024-09-16T22:16:26.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5333e-05 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154517+0000", 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4227000000000003e-05 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13335 3.14 3:2c852cac:::rbd_data.11cdc482f8dc.000000000000017c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:26.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.63095877899999997, 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067530489999999997, 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13335 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147468+0000", 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0079999999999999e-06 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147474+0000", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147490+0000", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6348999999999999e-05 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147498+0000", 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6459999999999993e-06 2024-09-16T22:16:26.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147560+0000", 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999995e-05 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147657+0000", 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7127999999999996e-05 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154220+0000", 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065637960000000002 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13337 3.d 3:b6885ab0:::rbd_data.11cdc482f8dc.0000000000000176:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.627821976, 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066670519999999997, 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13337 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150605+0000", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0999999999999998e-07 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150609+0000", 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9600000000000002e-06 2024-09-16T22:16:26.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150618+0000", 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4690000000000003e-06 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150626+0000", 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0630000000000006e-06 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150678+0000", 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1576000000000002e-05 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150751+0000", 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2941999999999994e-05 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150950+0000", 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019875399999999999 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157112+0000", 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061624089999999998 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157190+0000", 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7756000000000006e-05 2024-09-16T22:16:26.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157236+0000", 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157271+0000", 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5587000000000003e-05 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13338 3.11 3:8d0fdcbe:::rbd_data.11cdc482f8dc.000000000000017a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62777694399999995, 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063968640000000004, 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13338 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150650+0000", 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3499999999999996e-07 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150653+0000", 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1429999999999998e-06 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150661+0000", 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8220000000000007e-06 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150665+0000", 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3440000000000003e-06 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150714+0000", 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8483999999999999e-05 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150791+0000", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7576000000000004e-05 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150995+0000", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020399700000000001 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154972+0000", 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039767350000000003 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.156963+0000", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019913079999999998 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157011+0000", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7568999999999998e-05 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157046+0000", 2024-09-16T22:16:26.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5351000000000001e-05 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13389 3.11 3:89f76da2:::rbd_data.11cdc482f8dc.0000000000000195:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.60185321000000003, 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069377570000000001, 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13389 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176574+0000", 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3e-06 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176579+0000", 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6110000000000004e-06 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176592+0000", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2721000000000001e-05 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176599+0000", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7149999999999998e-06 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176657+0000", 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7246e-05 2024-09-16T22:16:26.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176788+0000", 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131187 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183511+0000", 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067229769999999998 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13396 3.a 3:57e20437:::rbd_data.11cdc482f8dc.000000000000019a:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.59647987800000002, 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065086880000000003, 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13396 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181947+0000", 2024-09-16T22:16:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4099999999999997e-07 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181952+0000", 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4170000000000002e-06 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181963+0000", 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1004e-05 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181968+0000", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182026+0000", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8622999999999998e-05 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182033+0000", 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184627+0000", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025939729999999999 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184764+0000", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013687500000000001 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188455+0000", 2024-09-16T22:16:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003690891 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13402 3.11 3:89f76da2:::rbd_data.11cdc482f8dc.0000000000000195:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.59465527600000001, 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081288179999999995, 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13402 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183772+0000", 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3820000000000001e-06 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183778+0000", 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3380000000000004e-06 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183789+0000", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1488000000000001e-05 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183797+0000", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3799999999999996e-06 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183849+0000", 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2092000000000002e-05 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183854+0000", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7019999999999998e-06 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184941+0000", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001086694 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185016+0000", 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4517000000000005e-05 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185208+0000", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191969 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.187389+0000", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021816829999999998 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191774+0000", 2024-09-16T22:16:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043850979999999996 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191843+0000", 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8522999999999994e-05 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191900+0000", 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6952e-05 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13403 3.14 3:2f33ca34:::rbd_data.11cdc482f8dc.0000000000000198:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.59440912499999998, 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081506990000000008, 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13403 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184018+0000", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0869999999999999e-06 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184024+0000", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184035+0000", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090000000000001e-05 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184042+0000", 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1280000000000002e-06 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184245+0000", 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000202618 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184250+0000", 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1390000000000003e-06 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185333+0000", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010835700000000001 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185402+0000", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8153999999999998e-05 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185587+0000", 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000185842 2024-09-16T22:16:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191939+0000", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063511710000000001 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192072+0000", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133573 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192130+0000", 2024-09-16T22:16:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8004000000000002e-05 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192168+0000", 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7574000000000003e-05 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13405 3.a 3:57e20437:::rbd_data.11cdc482f8dc.000000000000019a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58966098700000003, 2024-09-16T22:16:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091332259999999995, 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13405 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188767+0000", 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4720000000000001e-06 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188772+0000", 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7359999999999998e-06 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188787+0000", 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4790000000000001e-05 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188795+0000", 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3440000000000003e-06 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188842+0000", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7547999999999999e-05 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188944+0000", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101483 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.189159+0000", 2024-09-16T22:16:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000215439 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194573+0000", 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054140380000000004 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197660+0000", 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003086891 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197855+0000", 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019547400000000001 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197898+0000", 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3010999999999999e-05 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13582 3.a 3:55e8ea82:::rbd_data.11cdc482f8dc.00000000000001f5:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.51387775499999999, 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069540649999999997, 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13582 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264549+0000", 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264553+0000", 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5530000000000001e-06 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264560+0000", 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4869999999999996e-06 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264568+0000", 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7060000000000004e-06 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264623+0000", 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4938000000000002e-05 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264699+0000", 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5827000000000004e-05 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264893+0000", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019441 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.268738+0000", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038447139999999999 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271425+0000", 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026871059999999999 2024-09-16T22:16:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271469+0000", 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3989999999999997e-05 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271503+0000", 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3816000000000003e-05 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:26.897 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:26.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:26.899 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:26.900+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:26.999 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:27.001+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:27.006 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:27.006 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:27.100 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:27.101+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:27.117 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:27.117 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:27.117 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:27.117 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:27.201 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:27.203+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:27.231 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.231 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:27.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13034 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16072704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.15856019, 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020645562999999999, 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13034 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.232 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6100000000000002e-07 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073761+0000", 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.613e-06 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073765+0000", 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073767+0000", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3929999999999998e-06 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094286+0000", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020518814 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094332+0000", 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6644999999999998e-05 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094402+0000", 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9738000000000002e-05 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13035 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16109568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1585403249999999, 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020762438000000001, 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13035 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073777+0000", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5499999999999999e-07 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073778+0000", 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2449999999999999e-06 2024-09-16T22:16:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073782+0000", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7189999999999999e-06 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073784+0000", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3989999999999998e-06 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094434+0000", 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020649743000000002 2024-09-16T22:16:27.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094497+0000", 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2924999999999998e-05 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094539+0000", 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2151999999999999e-05 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13036 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1585292380000001, 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020877289, 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13036 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073789+0000", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7300000000000005e-07 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073792+0000", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9009999999999998e-06 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073794+0000", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2570000000000002e-06 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094581+0000", 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020786782 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094622+0000", 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1826999999999998e-05 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094665+0000", 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2348999999999997e-05 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13037 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158520126, 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020988652, 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13037 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0699999999999999e-07 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073798+0000", 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1399999999999996e-07 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073801+0000", 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8459999999999999e-06 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073803+0000", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.373e-06 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094701+0000", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020898110000000001 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094745+0000", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4134999999999997e-05 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094785+0000", 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0166999999999997e-05 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13038 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158511149, 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021115722999999999, 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13038 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1e-07 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073807+0000", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0699999999999996e-07 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073810+0000", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4000000000000001e-06 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073812+0000", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094818+0000", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021005830999999999 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094881+0000", 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3137000000000002e-05 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094921+0000", 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0015000000000002e-05 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13039 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16257024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158501647, 2024-09-16T22:16:27.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021219551999999999, 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13039 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073816+0000", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073819+0000", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9059999999999998e-06 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073822+0000", 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.977e-06 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094953+0000", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021130830999999999 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094992+0000", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9314999999999999e-05 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095035+0000", 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2514e-05 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13040 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16293888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1584920190000001, 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021350451999999999, 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13040 2024-09-16T22:16:27.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073826+0000", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0800000000000004e-07 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073833+0000", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.866e-06 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073835+0000", 2024-09-16T22:16:27.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4030000000000001e-06 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095068+0000", 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021233431000000001 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095104+0000", 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5255e-05 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095175+0000", 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1475000000000003e-05 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13041 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16330752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158478356, 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021462193000000001, 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13041 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073839+0000", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4400000000000001e-07 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073843+0000", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5449999999999997e-06 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073847+0000", 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-16T22:16:27.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073849+0000", 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095216+0000", 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021367077000000002 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095258+0000", 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2027999999999998e-05 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095301+0000", 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2323000000000001e-05 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13042 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1584638190000001, 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021557522999999999, 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13042 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.36e-07 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073854+0000", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-07 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073857+0000", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9620000000000002e-06 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073863+0000", 2024-09-16T22:16:27.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8619999999999999e-06 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095328+0000", 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021464589999999999 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095367+0000", 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9867999999999998e-05 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095410+0000", 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3102999999999997e-05 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13043 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16404480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158437363, 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021632377000000001, 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13043 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:27.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073880+0000", 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073881+0000", 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3179999999999999e-06 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073885+0000", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6729999999999998e-06 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073887+0000", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4590000000000001e-06 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095436+0000", 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021548233999999999 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095473+0000", 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7478000000000003e-05 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095512+0000", 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804000000000003e-05 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13044 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.275 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1584263029999999, 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021719948999999999, 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13044 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073891+0000", 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073895+0000", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9829999999999998e-06 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073898+0000", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1729999999999999e-06 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073900+0000", 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.345e-06 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095535+0000", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021634322000000001 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095571+0000", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095610+0000", 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9387000000000001e-05 2024-09-16T22:16:27.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13045 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16478208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1584135200000001, 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021805422000000001, 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13045 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073904+0000", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9999999999999996e-07 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073907+0000", 2024-09-16T22:16:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9019999999999999e-06 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073910+0000", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8449999999999999e-06 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095633+0000", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021722769999999999 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095672+0000", 2024-09-16T22:16:27.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8688e-05 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095709+0000", 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7097000000000001e-05 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13046 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:27.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1584023020000001, 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021886407, 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13046 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:27.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073915+0000", 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0800000000000001e-07 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073916+0000", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6400000000000001e-07 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073919+0000", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8410000000000002e-06 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073923+0000", 2024-09-16T22:16:27.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1259999999999998e-06 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095729+0000", 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021806635000000001 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095765+0000", 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5670999999999998e-05 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095801+0000", 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6062000000000001e-05 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13047 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16551936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1583907200000001, 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021980845999999998, 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13047 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:27.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.11e-07 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073927+0000", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2500000000000004e-07 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073930+0000", 2024-09-16T22:16:27.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.943e-06 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073935+0000", 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0579999999999997e-06 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095823+0000", 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021887747999999999 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095867+0000", 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4206000000000003e-05 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095907+0000", 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9854999999999997e-05 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13048 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16588800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1583781710000001, 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022059216, 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13048 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.17e-07 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073940+0000", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3099999999999996e-07 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073943+0000", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9919999999999999e-06 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073945+0000", 2024-09-16T22:16:27.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095922+0000", 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021977272999999999 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095959+0000", 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6473000000000001e-05 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095998+0000", 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9183000000000001e-05 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13049 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16625664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158369108, 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022143023000000001, 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13049 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0800000000000001e-07 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073949+0000", 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1800000000000004e-07 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073952+0000", 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.393e-06 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073955+0000", 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3470000000000001e-06 2024-09-16T22:16:27.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096014+0000", 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022059055000000001 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096052+0000", 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8871000000000001e-05 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096091+0000", 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8231e-05 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13050 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1583443200000001, 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022205784999999999, 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13050 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073973+0000", 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8700000000000001e-07 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073974+0000", 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2109999999999999e-06 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073977+0000", 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.067e-06 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073980+0000", 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3920000000000001e-06 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096104+0000", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022124516 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096140+0000", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5537000000000002e-05 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096178+0000", 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8674999999999998e-05 2024-09-16T22:16:27.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13051 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16699392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1583337710000001, 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022283160999999999, 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13051 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073984+0000", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.16e-07 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073987+0000", 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9019999999999999e-06 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073994+0000", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.221e-06 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096193+0000", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022198401 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096230+0000", 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7231000000000003e-05 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096266+0000", 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6180999999999998e-05 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13052 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16736256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.158319315, 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022365262, 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13052 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:27.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073998+0000", 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073999+0000", 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3600000000000002e-07 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074004+0000", 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8900000000000004e-06 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074007+0000", 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096290+0000", 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022282381 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096326+0000", 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5824000000000001e-05 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096363+0000", 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7184999999999997e-05 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.316 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13053 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16773120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1583063060000001, 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022442749000000001, 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13053 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074011+0000", 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.16e-07 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074014+0000", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074017+0000", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2310000000000001e-06 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074020+0000", 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3439999999999999e-06 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096379+0000", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022359875000000001 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096416+0000", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6431999999999999e-05 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096453+0000", 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7385999999999997e-05 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.321 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:27.322 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:27.323+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:27.422 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:27.424+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:27.472 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:27.472 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:27.484 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T22:16:27.523 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:27.525+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:27.624 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.624 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:27.624 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:27.624 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:27.624 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.625 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:27.626+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:27.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:558 read 0xb55f9f thru 0xb5d4ba (0x751c bytes) 2024-09-16T22:16:27.636 INFO:teuthology.orchestra.run.smithi031.stdout:560 punch from 0x674abe7 to 0x674ac4a, (0x63 bytes) 2024-09-16T22:16:27.639 INFO:teuthology.orchestra.run.smithi031.stdout:561 punch from 0x32026b6 to 0x3208d10, (0x665a bytes) 2024-09-16T22:16:27.639 INFO:teuthology.orchestra.run.smithi031.stdout:563 read 0x7fbdb3e thru 0x7fbfeaf (0x2372 bytes) 2024-09-16T22:16:27.639 INFO:teuthology.orchestra.run.smithi031.stdout:564 read 0x4f160f2 thru 0x4f161b7 (0xc6 bytes) 2024-09-16T22:16:27.639 INFO:teuthology.orchestra.run.smithi031.stdout:565 write 0x38273a5 thru 0x383171a (0xa376 bytes) 2024-09-16T22:16:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:567 trunc from 0x803e62e to 0x7c295 2024-09-16T22:16:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.724 INFO:teuthology.orchestra.run.smithi031.stdout:570 trunc from 0x7c295 to 0xb8cf168 2024-09-16T22:16:27.724 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:27.726+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:27.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:27.729 INFO:teuthology.orchestra.run.smithi031.stdout:574 write 0x7e248b6 thru 0x7e264c0 (0x1c0b bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:576 read 0x8fe1bb1 thru 0x8feae20 (0x9270 bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:577 read 0x3b0ca12 thru 0x3b1a514 (0xdb03 bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:578 punch from 0x2ba9bfd to 0x2bae9e1, (0x4de4 bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:579 punch from 0x7d9e44a to 0x7da31b8, (0x4d6e bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:580 read 0x694ec4d thru 0x6954440 (0x57f4 bytes) 2024-09-16T22:16:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:581 read 0x8b91c40 thru 0x8b9a0b0 (0x8471 bytes) 2024-09-16T22:16:27.740 INFO:teuthology.orchestra.run.smithi031.stdout:582 writesame 0xbfc5714 thru 0xbfcd595 data_size 0x1(0x7e82 bytes) 2024-09-16T22:16:27.740 INFO:teuthology.orchestra.run.smithi031.stdout:583 punch from 0x5ff9b to 0x64be1, (0x4c46 bytes) 2024-09-16T22:16:27.740 INFO:teuthology.orchestra.run.smithi031.stdout:584 punch from 0x40ec4ce to 0x40f40d0, (0x7c02 bytes) 2024-09-16T22:16:27.740 INFO:teuthology.orchestra.run.smithi031.stdout:585 write 0x3aaa467 thru 0x3ab3e79 (0x9a13 bytes) 2024-09-16T22:16:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:586 trunc from 0xbfcd596 to 0xd167206 2024-09-16T22:16:27.749 INFO:teuthology.orchestra.run.smithi031.stdout:587 read 0xb6fd5ce thru 0xb70ae7a (0xd8ad bytes) 2024-09-16T22:16:27.749 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.749 INFO:teuthology.orchestra.run.smithi031.stdout:593 read 0x946798b thru 0x9468f34 (0x15aa bytes) 2024-09-16T22:16:27.750 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:27.750 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:27.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:42 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'24, mlcod=17'19)", 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.219129583000001, 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056317980000000004, 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532088+0000", 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532089+0000", 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3300000000000001e-07 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532096+0000", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0450000000000003e-06 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532106+0000", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532111+0000", 2024-09-16T22:16:27.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2950000000000001e-06 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532156+0000", 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4672000000000002e-05 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.532171+0000", 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5392000000000001e-05 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537693+0000", 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055218339999999998 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:58.537720+0000", 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7285e-05 2024-09-16T22:16:27.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:90 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=17'60)", 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.621115515, 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056948570000000002, 2024-09-16T22:16:27.756 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130103+0000", 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130163+0000", 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0516999999999997e-05 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130172+0000", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3990000000000004e-06 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130177+0000", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8960000000000002e-06 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130210+0000", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3492999999999999e-05 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130220+0000", 2024-09-16T22:16:27.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0176999999999999e-05 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135746+0000", 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055257730000000003 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135797+0000", 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0982000000000001e-05 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.616329090000001, 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095371610000000006, 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 11 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134890+0000", 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.556e-06 2024-09-16T22:16:27.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134902+0000", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1999e-05 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134914+0000", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1233000000000001e-05 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134919+0000", 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.48e-06 2024-09-16T22:16:27.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134968+0000", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8915000000000002e-05 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134972+0000", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7929999999999999e-06 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135632+0000", 2024-09-16T22:16:27.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065986099999999998 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135741+0000", 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109416 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135889+0000", 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014819200000000001 2024-09-16T22:16:27.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.137526+0000", 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001637129 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144330+0000", 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068036010000000003 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144379+0000", 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9141999999999998e-05 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144426+0000", 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6844000000000001e-05 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1726 3.18 3:1ec1f583:::rbd_data.11cd70c2eac3.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.563623+0000", 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1875953040000002, 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056277380000000002, 2024-09-16T22:16:27.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1726 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563623+0000", 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563623+0000", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563623+0000", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563627+0000", 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.765e-06 2024-09-16T22:16:27.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563637+0000", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4769999999999992e-06 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563658+0000", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1916000000000002e-05 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563706+0000", 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7682000000000002e-05 2024-09-16T22:16:27.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563755+0000", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8510000000000001e-05 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.563818+0000", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3436999999999996e-05 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.566103+0000", 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002284945 2024-09-16T22:16:27.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.569160+0000", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003057269 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.569213+0000", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2524000000000001e-05 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.569250+0000", 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7437000000000001e-05 2024-09-16T22:16:27.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1894 3.0 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.9560218029999996, 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006210162, 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1894 2024-09-16T22:16:27.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795197+0000", 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795202+0000", 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7190000000000001e-06 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795215+0000", 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3383e-05 2024-09-16T22:16:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795223+0000", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8450000000000003e-06 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795287+0000", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3838999999999996e-05 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795368+0000", 2024-09-16T22:16:27.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0927000000000006e-05 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795486+0000", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011838800000000001 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.796188+0000", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070161500000000005 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801350+0000", 2024-09-16T22:16:27.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051619099999999996 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801379+0000", 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9549000000000001e-05 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801406+0000", 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6996999999999999e-05 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1916 3.0 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.9202698229999999, 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015511264, 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1916 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830949+0000", 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830953+0000", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.476e-06 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830964+0000", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529e-05 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830969+0000", 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2519999999999999e-06 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.831008+0000", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8915999999999998e-05 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.831057+0000", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9146e-05 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.833521+0000", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024641960000000001 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846354+0000", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012832427 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846392+0000", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8724999999999999e-05 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846428+0000", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5593000000000003e-05 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846459+0000", 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1269999999999997e-05 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:8552 3.17 3:ee12f636:::rbd_data.11cdc482f8dc.000000000000007e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.075179849, 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006238637, 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 8552 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676039+0000", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676046+0000", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4189999999999999e-06 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676058+0000", 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2566e-05 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676066+0000", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3650000000000004e-06 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676116+0000", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9981000000000001e-05 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676178+0000", 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1494000000000004e-05 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676390+0000", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021231900000000001 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676824+0000", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043400399999999999 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682223+0000", 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053988350000000003 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682255+0000", 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1813000000000002e-05 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682277+0000", 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1661000000000001e-05 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9869 3.1e 3:7faa808b:::rbd_data.11cdc482f8dc.00000000000000b2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.94722131, 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068475999999999997, 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9869 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0500000000000002e-07 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804003+0000", 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.169e-06 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804015+0000", 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2856e-05 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804023+0000", 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5900000000000002e-06 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804069+0000", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5568999999999997e-05 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804115+0000", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6455000000000002e-05 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804238+0000", 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122967 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804710+0000", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000472483 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810720+0000", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060092699999999997 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810805+0000", 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4958999999999998e-05 2024-09-16T22:16:27.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810844+0000", 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9376999999999999e-05 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9900 3.1e 3:7fed7f31:::rbd_data.11cdc482f8dc.00000000000000c5:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.929551963, 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058095239999999999, 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9900 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821667+0000", 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3200000000000004e-07 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821671+0000", 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0840000000000002e-06 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821680+0000", 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.799e-06 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821685+0000", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821722+0000", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7854e-05 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821792+0000", 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0006999999999996e-05 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.827475+0000", 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056828929999999996 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:10823 3.1a e31/23 3:5ab4cea0:::rbd_data.11cdc482f8dc.00000000000000ca:head v 31'52, mlcod=31'50)", 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.843716659, 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005731603, 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907502+0000", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1099999999999995e-07 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907507+0000", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2279999999999998e-06 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907519+0000", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907526+0000", 2024-09-16T22:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.55e-06 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907580+0000", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907602+0000", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2166000000000001e-05 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.913191+0000", 2024-09-16T22:16:27.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055893219999999999 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.913233+0000", 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1476000000000001e-05 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13229 3.1a e31/23 3:582e74af:::rbd_data.11cdc482f8dc.0000000000000120:head v 31'63, mlcod=31'61)", 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.640032113, 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058645540000000001, 2024-09-16T22:16:27.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111187+0000", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111192+0000", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111203+0000", 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0560000000000001e-05 2024-09-16T22:16:27.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111209+0000", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4479999999999999e-06 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111270+0000", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0690999999999999e-05 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111290+0000", 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0163999999999999e-05 2024-09-16T22:16:27.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.117009+0000", 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057184690000000003 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.117050+0000", 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1517999999999998e-05 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13239 3.18 3:1836d7a2:::rbd_data.11cdc482f8dc.000000000000014c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.636782564, 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057377039999999997, 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13239 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114436+0000", 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114443+0000", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7320000000000001e-06 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114453+0000", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9110000000000005e-06 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114459+0000", 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.798e-06 2024-09-16T22:16:27.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114508+0000", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8621000000000001e-05 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114628+0000", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120402 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120173+0000", 2024-09-16T22:16:27.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005544546 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13257 3.18 3:1836d7a2:::rbd_data.11cdc482f8dc.000000000000014c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6303962269999999, 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075231200000000003, 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13257 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120826+0000", 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9060000000000004e-06 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120842+0000", 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5800000000000001e-05 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120847+0000", 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8709999999999996e-06 2024-09-16T22:16:27.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120887+0000", 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0488000000000003e-05 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120941+0000", 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3610999999999999e-05 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:27.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.121042+0000", 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101389 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.125714+0000", 2024-09-16T22:16:27.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046714859999999999 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128169+0000", 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024548880000000001 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128307+0000", 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013875700000000001 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128345+0000", 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7304000000000001e-05 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13258 3.1e 3:7eafc415:::rbd_data.11cdc482f8dc.0000000000000154:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6303349410000001, 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074323890000000002, 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13258 2024-09-16T22:16:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120886+0000", 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1550000000000001e-06 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120890+0000", 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7620000000000003e-06 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120894+0000", 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3720000000000001e-06 2024-09-16T22:16:27.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120934+0000", 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0088e-05 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120999+0000", 2024-09-16T22:16:27.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104000000000006e-05 2024-09-16T22:16:27.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128315+0000", 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073164140000000003 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13378 3.0 3:06bf120b:::rbd_data.11cdc482f8dc.000000000000018b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.5805102230000001, 2024-09-16T22:16:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057724650000000001, 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13378 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:27.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170709+0000", 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0189999999999999e-06 2024-09-16T22:16:27.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170714+0000", 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170727+0000", 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.274e-05 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170734+0000", 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6159999999999996e-06 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170783+0000", 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8437999999999999e-05 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170843+0000", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0201000000000002e-05 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170958+0000", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114961 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.173462+0000", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002503917 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176392+0000", 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029305030000000001 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176443+0000", 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0800000000000002e-05 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176480+0000", 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6950999999999999e-05 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13387 3.1e 3:7b9a15c0:::rbd_data.11cdc482f8dc.0000000000000190:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.5760549579999998, 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080670580000000002, 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13387 2024-09-16T22:16:27.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175164+0000", 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6000000000000002e-07 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175171+0000", 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175182+0000", 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0926e-05 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175187+0000", 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175224+0000", 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6868e-05 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175269+0000", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.515e-05 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175374+0000", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104876 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179182+0000", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003808287 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183166+0000", 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039836710000000003 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183203+0000", 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200999999999998e-05 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183230+0000", 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7151000000000001e-05 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13434 3.17 3:ea1a1df2:::rbd_data.11cdc482f8dc.00000000000001a8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.5459703039999999, 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060193850000000004, 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13434 2024-09-16T22:16:27.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5199999999999995e-07 2024-09-16T22:16:27.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205253+0000", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7169999999999999e-06 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205265+0000", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2216e-05 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205273+0000", 2024-09-16T22:16:27.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6580000000000002e-06 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205323+0000", 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0488999999999997e-05 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205395+0000", 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1654000000000002e-05 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:27.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205579+0000", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183926 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206631+0000", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010517129999999999 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211212+0000", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045817269999999998 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211244+0000", 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1452999999999999e-05 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211267+0000", 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.298e-05 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13454 3.17 3:ed04fddc:::rbd_data.11cdc482f8dc.00000000000001b3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.539313326, 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073096990000000002, 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13454 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211905+0000", 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211909+0000", 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0210000000000001e-06 2024-09-16T22:16:27.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211917+0000", 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211923+0000", 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9800000000000003e-06 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211973+0000", 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9599999999999999e-05 2024-09-16T22:16:27.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212040+0000", 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7057999999999994e-05 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:27.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212241+0000", 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020084200000000001 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.213002+0000", 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076082000000000005 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219131+0000", 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061289140000000001 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219179+0000", 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8312000000000001e-05 2024-09-16T22:16:27.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219214+0000", 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5275999999999999e-05 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.10 epoch 32/23 rep_tid 221 entries 32'35 (0'0) error 3:08d88565:::rbd_data.11cdc482f8dc.00000000000001b8:head by client.4557.0:13455 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'33)", 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.5387941330000001, 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067463319999999998, 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4699999999999998e-07 2024-09-16T22:16:27.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212429+0000", 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212447+0000", 2024-09-16T22:16:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7986e-05 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212461+0000", 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4149e-05 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212542+0000", 2024-09-16T22:16:27.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0692000000000006e-05 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219170+0000", 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066281530000000003 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13471 3.1e 3:788c3ff6:::rbd_data.11cdc482f8dc.00000000000001c0:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.531443063, 2024-09-16T22:16:27.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056951670000000001, 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13471 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:27.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219776+0000", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1200000000000001e-07 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219780+0000", 2024-09-16T22:16:27.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2309999999999998e-06 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219790+0000", 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0122000000000001e-05 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219797+0000", 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9e-06 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219862+0000", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5093999999999997e-05 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219989+0000", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012717100000000001 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225470+0000", 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054807370000000003 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:27.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:27.876 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:27.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:27.878 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:27.878+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:595 writesame 0xde5f833 thru 0xde6f59f data_size 0x1(0xfd6d bytes) 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:596 write 0x883a2ee thru 0x8848f5f (0xec72 bytes) 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:597 read 0x394bfe2 thru 0x39569de (0xa9fd bytes) 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:598 punch from 0x56afaac to 0x56b17e5, (0x1d39 bytes) 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:601 trunc from 0xde6f5a0 to 0x62de5ac 2024-09-16T22:16:27.881 INFO:teuthology.orchestra.run.smithi031.stdout:603 writesame 0x6b27230 thru 0x6b343ad data_size 0x9b(0xd17e bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:604 read 0x5e64183 thru 0x5e6ca79 (0x88f7 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:605 writesame 0x5a97620 thru 0x5a9cd25 data_size 0x6(0x5706 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:606 writesame 0x98bbc0b thru 0x98bedc4 data_size 0x43(0x31ba bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:607 write 0xe9706df thru 0xe979bf1 (0x9513 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:610 punch from 0xac3f60d to 0xac4465c, (0x504f bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:613 write 0x93e7d2b thru 0x93e9cae (0x1f84 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:616 read 0xd06eca7 thru 0xd07c487 (0xd7e1 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:617 punch from 0x1554a6e to 0x1555772, (0xd04 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:623 read 0x93f3c21 thru 0x93fa1b2 (0x6592 bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:624 read 0xbed8efa thru 0xbedcff4 (0x40fb bytes) 2024-09-16T22:16:27.882 INFO:teuthology.orchestra.run.smithi031.stdout:625 trunc from 0xe979bf2 to 0xd461d98 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:626 writesame 0x6111c99 thru 0x6117a01 data_size 0x3(0x5d69 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:627 read 0x2e8f4a8 thru 0x2e96b98 (0x76f1 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:628 read 0xc0e6aa8 thru 0xc0e773d (0xc96 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:629 read 0xbde1378 thru 0xbde6f0a (0x5b93 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:632 trunc from 0xd461d98 to 0xde5e227 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:633 write 0x9b1ae0c thru 0x9b2966c (0xe861 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:634 write 0x54d8178 thru 0x54dbfc3 (0x3e4c bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:635 writesame 0x26a0f13 thru 0x26a62eb data_size 0x51(0x53d9 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:638 trunc from 0xde5e227 to 0xc00c00f 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:640 read 0xa0ad8c8 thru 0xa0ae0ed (0x826 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:641 read 0x736548d thru 0x73741ec (0xed60 bytes) 2024-09-16T22:16:27.883 INFO:teuthology.orchestra.run.smithi031.stdout:642 punch from 0x1a9afe0 to 0x1aa8d16, (0xdd36 bytes) 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stdout:643 writesame 0x6324569 thru 0x6330770 data_size 0x1c(0xc208 bytes) 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stdout:644 write 0x90945c8 thru 0x9099f21 (0x595a bytes) 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stdout:645 writesame 0x1ccf5de thru 0x1cdb17b data_size 0xf(0xbb9e bytes) 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stdout:647 write 0x6e2f773 thru 0x6e300a8 (0x936 bytes) 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:648 write 0x23af822 thru 0x23bbe42 (0xc621 bytes) 2024-09-16T22:16:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:649 read 0xa0fe47 thru 0xa11ed5 (0x208f bytes) 2024-09-16T22:16:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:650 read 0x6a4987c thru 0x6a4d2c5 (0x3a4a bytes) 2024-09-16T22:16:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:651 write 0x331f699 thru 0x3322f5f (0x38c7 bytes) 2024-09-16T22:16:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:653 trunc from 0xc00c00f to 0xf0488a 2024-09-16T22:16:27.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:27.896 INFO:teuthology.orchestra.run.smithi031.stdout:654 read 0x40f629 thru 0x41bc4f (0xc627 bytes) 2024-09-16T22:16:27.919 INFO:teuthology.orchestra.run.smithi031.stdout:655 write 0xe475dbf thru 0xe47c6c3 (0x6905 bytes) 2024-09-16T22:16:27.922 INFO:teuthology.orchestra.run.smithi031.stdout:656 punch from 0xd3e93a1 to 0xd3ee42c, (0x508b bytes) 2024-09-16T22:16:27.922 INFO:teuthology.orchestra.run.smithi031.stdout:659 punch from 0x3098547 to 0x3099337, (0xdf0 bytes) 2024-09-16T22:16:27.922 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.922 INFO:teuthology.orchestra.run.smithi031.stdout:661 write 0x33007 thru 0x36e97 (0x3e91 bytes) 2024-09-16T22:16:27.923 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:27.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:27.923 INFO:teuthology.orchestra.run.smithi031.stdout:662 read 0x4699023 thru 0x469fc37 (0x6c15 bytes) 2024-09-16T22:16:27.923 INFO:teuthology.orchestra.run.smithi031.stdout:663 writesame 0xcd35656 thru 0xcd373b0 data_size 0xf(0x1d5b bytes) 2024-09-16T22:16:27.926 INFO:teuthology.orchestra.run.smithi031.stdout:665 writesame 0x803c270 thru 0x8046078 data_size 0x1(0x9e09 bytes) 2024-09-16T22:16:27.940 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:27.948 INFO:teuthology.orchestra.run.smithi031.stdout:666 punch from 0x6c73cfc to 0x6c7b373, (0x7677 bytes) 2024-09-16T22:16:27.948 INFO:teuthology.orchestra.run.smithi031.stdout:668 read 0x1226dff thru 0x12304d0 (0x96d2 bytes) 2024-09-16T22:16:27.948 INFO:teuthology.orchestra.run.smithi031.stdout:671 read 0x4f4249 thru 0x4fa303 (0x60bb bytes) 2024-09-16T22:16:27.948 INFO:teuthology.orchestra.run.smithi031.stdout:672 write 0x6744145 thru 0x674c1c0 (0x807c bytes) 2024-09-16T22:16:27.951 INFO:teuthology.orchestra.run.smithi031.stdout:673 read 0x1fbe51e thru 0x1fc5479 (0x6f5c bytes) 2024-09-16T22:16:27.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:27.958 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:27.958 INFO:teuthology.orchestra.run.smithi031.stdout:674 trunc from 0xe47c6c4 to 0x8b1a0c8 2024-09-16T22:16:27.967 INFO:teuthology.orchestra.run.smithi031.stdout:675 read 0x64cf562 thru 0x64d1460 (0x1eff bytes) 2024-09-16T22:16:27.967 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.968 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.968 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:27.968 INFO:teuthology.orchestra.run.smithi031.stdout:679 read 0x21b9e1a thru 0x21c2699 (0x8880 bytes) 2024-09-16T22:16:27.968 INFO:teuthology.orchestra.run.smithi031.stdout:680 writesame 0xde524a thru 0xde8ba1 data_size 0x14(0x3958 bytes) 2024-09-16T22:16:27.970 INFO:teuthology.orchestra.run.smithi031.stdout:681 write 0x4827aaa thru 0x482a561 (0x2ab8 bytes) 2024-09-16T22:16:27.973 INFO:teuthology.orchestra.run.smithi031.stdout:682 read 0x728b4ab thru 0x7296c77 (0xb7cd bytes) 2024-09-16T22:16:27.973 INFO:teuthology.orchestra.run.smithi031.stdout:684 write 0x3e5282c thru 0x3e60e73 (0xe648 bytes) 2024-09-16T22:16:27.976 INFO:teuthology.orchestra.run.smithi031.stdout:685 read 0xcfd920 thru 0xd05fef (0x86d0 bytes) 2024-09-16T22:16:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:687 trunc from 0x8b1a0c8 to 0x82753bf 2024-09-16T22:16:27.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:27.977 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:27.978 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:27.979+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:27.983 INFO:teuthology.orchestra.run.smithi031.stdout:688 write 0x1713ec1 thru 0x1718b8d (0x4ccd bytes) 2024-09-16T22:16:27.984 INFO:teuthology.orchestra.run.smithi031.stdout:689 writesame 0x36f34cb thru 0x36fa4bd data_size 0x29(0x6ff3 bytes) 2024-09-16T22:16:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:691 read 0x176e0e8 thru 0x1775b45 (0x7a5e bytes) 2024-09-16T22:16:27.993 INFO:teuthology.orchestra.run.smithi031.stdout:692 writesame 0x8b0b2f8 thru 0x8b1635a data_size 0xb(0xb063 bytes) 2024-09-16T22:16:27.993 INFO:teuthology.orchestra.run.smithi031.stdout:693 write 0x4301948 thru 0x43075d1 (0x5c8a bytes) 2024-09-16T22:16:27.996 INFO:teuthology.orchestra.run.smithi031.stdout:694 write 0x7403fb3 thru 0x74125b7 (0xe605 bytes) 2024-09-16T22:16:27.997 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:27.997 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:28.000 INFO:teuthology.orchestra.run.smithi031.stdout:696 trunc from 0x8b1635b to 0xa193d54 2024-09-16T22:16:28.005 INFO:teuthology.orchestra.run.smithi031.stdout:697 write 0x8533dcb thru 0x85419a9 (0xdbdf bytes) 2024-09-16T22:16:28.015 INFO:teuthology.orchestra.run.smithi031.stdout:698 writesame 0xb6b94b7 thru 0xb6b9697 data_size 0x1(0x1e1 bytes) 2024-09-16T22:16:28.015 INFO:teuthology.orchestra.run.smithi031.stdout:700 read 0x67902b7 thru 0x6796467 (0x61b1 bytes) 2024-09-16T22:16:28.015 INFO:teuthology.orchestra.run.smithi031.stdout:701 writesame 0x9568d80 thru 0x9577323 data_size 0x8a(0xe5a4 bytes) 2024-09-16T22:16:28.015 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:28.016 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:28.016 INFO:teuthology.orchestra.run.smithi031.stdout:702 read 0x10f0e54 thru 0x10fe1eb (0xd398 bytes) 2024-09-16T22:16:28.017 INFO:teuthology.orchestra.run.smithi031.stdout:704 write 0x8be2d2b thru 0x8bf1724 (0xe9fa bytes) 2024-09-16T22:16:28.020 INFO:teuthology.orchestra.run.smithi031.stdout:705 write 0x251c8bc thru 0x252c4a4 (0xfbe9 bytes) 2024-09-16T22:16:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:707 write 0x459fec8 thru 0x45a03bc (0x4f5 bytes) 2024-09-16T22:16:28.029 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.030 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.031 INFO:teuthology.orchestra.run.smithi031.stdout:709 trunc from 0xb6b9698 to 0x57f0d89 2024-09-16T22:16:28.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:28.045 INFO:teuthology.orchestra.run.smithi031.stdout:710 write 0x23d484c thru 0x23de81b (0x9fd0 bytes) 2024-09-16T22:16:28.056 INFO:teuthology.orchestra.run.smithi031.stdout:711 trunc from 0x57f0d89 to 0xe11bfa2 2024-09-16T22:16:28.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.072 INFO:teuthology.orchestra.run.smithi031.stdout:715 trunc from 0xe11bfa2 to 0x473b966 2024-09-16T22:16:28.078 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:28.080+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:28.079 INFO:teuthology.orchestra.run.smithi031.stdout:716 write 0x3bba91c thru 0x3bc6327 (0xba0c bytes) 2024-09-16T22:16:28.082 INFO:teuthology.orchestra.run.smithi031.stdout:717 punch from 0x13c8688 to 0x13d2348, (0x9cc0 bytes) 2024-09-16T22:16:28.082 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.096 INFO:teuthology.orchestra.run.smithi031.stdout:720 writesame 0xac3eddd thru 0xac4015e data_size 0xb(0x1382 bytes) 2024-09-16T22:16:28.096 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.096 INFO:teuthology.orchestra.run.smithi031.stdout:723 write 0x3497a7f thru 0x34a526a (0xd7ec bytes) 2024-09-16T22:16:28.098 INFO:teuthology.orchestra.run.smithi031.stdout:724 writesame 0x8163e3 thru 0x816a65 data_size 0x1(0x683 bytes) 2024-09-16T22:16:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:725 read 0x1f5439b thru 0x1f59c3c (0x58a2 bytes) 2024-09-16T22:16:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:728 read 0x38d33e1 thru 0x38d4730 (0x1350 bytes) 2024-09-16T22:16:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:729 write 0x339f1dd thru 0x33ae289 (0xf0ad bytes) 2024-09-16T22:16:28.112 INFO:teuthology.orchestra.run.smithi031.stdout:730 trunc from 0xac4015f to 0x37b48d4 2024-09-16T22:16:28.124 INFO:teuthology.orchestra.run.smithi031.stdout:731 punch from 0x232bc6b to 0x23304ca, (0x485f bytes) 2024-09-16T22:16:28.125 INFO:teuthology.orchestra.run.smithi031.stdout:734 trunc from 0x37b48d4 to 0x22c580b 2024-09-16T22:16:28.142 INFO:teuthology.orchestra.run.smithi031.stdout:735 write 0x4732103 thru 0x473f1d4 (0xd0d2 bytes) 2024-09-16T22:16:28.145 INFO:teuthology.orchestra.run.smithi031.stdout:737 read 0x3f9ad47 thru 0x3f9e1cd (0x3487 bytes) 2024-09-16T22:16:28.145 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.145 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.146 INFO:teuthology.orchestra.run.smithi031.stdout:741 punch from 0x1fc3e67 to 0x1fc71fa, (0x3393 bytes) 2024-09-16T22:16:28.148 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.148 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:28.162 INFO:teuthology.orchestra.run.smithi031.stdout:742 write 0xd5b5de7 thru 0xd5b6095 (0x2af bytes) 2024-09-16T22:16:28.165 INFO:teuthology.orchestra.run.smithi031.stdout:744 read 0x60330c6 thru 0x603b6d5 (0x8610 bytes) 2024-09-16T22:16:28.166 INFO:teuthology.orchestra.run.smithi031.stdout:745 read 0x5981764 thru 0x5989c9f (0x853c bytes) 2024-09-16T22:16:28.166 INFO:teuthology.orchestra.run.smithi031.stdout:747 read 0x6dcbb19 thru 0x6dd16ff (0x5be7 bytes) 2024-09-16T22:16:28.166 INFO:teuthology.orchestra.run.smithi031.stdout:749 write 0x84fd7be thru 0x85041f8 (0x6a3b bytes) 2024-09-16T22:16:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:750 write 0xe7a9834 thru 0xe7ac350 (0x2b1d bytes) 2024-09-16T22:16:28.189 INFO:teuthology.orchestra.run.smithi031.stdout:751 trunc from 0xe7ac351 to 0x7a5820c 2024-09-16T22:16:28.199 INFO:teuthology.orchestra.run.smithi031.stdout:752 writesame 0x34ee391 thru 0x34fca7a data_size 0xb(0xe6ea bytes) 2024-09-16T22:16:28.202 INFO:teuthology.orchestra.run.smithi031.stdout:753 punch from 0x3fc764c to 0x3fc8b0b, (0x14bf bytes) 2024-09-16T22:16:28.202 INFO:teuthology.orchestra.run.smithi031.stdout:754 write 0x4e140e0 thru 0x4e1620c (0x212d bytes) 2024-09-16T22:16:28.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:16:28.204 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:28.204+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:28.218 INFO:teuthology.orchestra.run.smithi031.stdout:756 writesame 0xed971fe thru 0xed9adea data_size 0x17(0x3bed bytes) 2024-09-16T22:16:28.218 INFO:teuthology.orchestra.run.smithi031.stdout:757 punch from 0xc039727 to 0xc0427bf, (0x9098 bytes) 2024-09-16T22:16:28.232 INFO:teuthology.orchestra.run.smithi031.stdout:758 trunc from 0xed9adeb to 0x2142c06 2024-09-16T22:16:28.254 INFO:teuthology.orchestra.run.smithi031.stdout:759 write 0x6483436 thru 0x648b73a (0x8305 bytes) 2024-09-16T22:16:28.264 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.264 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:760 writesame 0xb240b9a thru 0xb241931 data_size 0x28(0xd98 bytes) 2024-09-16T22:16:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:763 read 0xe9e7a6 thru 0xeae056 (0xf8b1 bytes) 2024-09-16T22:16:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:765 write 0x623013 thru 0x630497 (0xd485 bytes) 2024-09-16T22:16:28.272 INFO:teuthology.orchestra.run.smithi031.stdout:766 write 0x5fef452 thru 0x5ff2b18 (0x36c7 bytes) 2024-09-16T22:16:28.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:28.275 INFO:teuthology.orchestra.run.smithi031.stdout:769 write 0xada6c11 thru 0xadabeb6 (0x52a6 bytes) 2024-09-16T22:16:28.277 INFO:teuthology.orchestra.run.smithi031.stdout:770 read 0x3686495 thru 0x3686d3d (0x8a9 bytes) 2024-09-16T22:16:28.278 INFO:teuthology.orchestra.run.smithi031.stdout:771 punch from 0x67fa372 to 0x6807048, (0xccd6 bytes) 2024-09-16T22:16:28.288 INFO:teuthology.orchestra.run.smithi031.stdout:772 trunc from 0xb241932 to 0x2180f0f 2024-09-16T22:16:28.299 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.299 INFO:teuthology.orchestra.run.smithi031.stdout:775 punch from 0x1bbd815 to 0x1bbef0d, (0x16f8 bytes) 2024-09-16T22:16:28.314 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:28.316+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:28.318 INFO:teuthology.orchestra.run.smithi031.stdout:777 write 0xdfd8b14 thru 0xdfdb5e3 (0x2ad0 bytes) 2024-09-16T22:16:28.321 INFO:teuthology.orchestra.run.smithi031.stdout:778 write 0xd7c9872 thru 0xd7d536c (0xbafb bytes) 2024-09-16T22:16:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:779 read 0xaf1e67a thru 0xaf28203 (0x9b8a bytes) 2024-09-16T22:16:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:780 read 0xc6b1c34 thru 0xc6b89f7 (0x6dc4 bytes) 2024-09-16T22:16:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:781 read 0x18f9f49 thru 0x18ff4e2 (0x559a bytes) 2024-09-16T22:16:28.339 INFO:teuthology.orchestra.run.smithi031.stdout:783 trunc from 0xdfdb5e4 to 0x15f2ca1 2024-09-16T22:16:28.350 INFO:teuthology.orchestra.run.smithi031.stdout:784 trunc from 0x15f2ca1 to 0x1277511 2024-09-16T22:16:28.367 INFO:teuthology.orchestra.run.smithi031.stdout:785 write 0x5a53b75 thru 0x5a61845 (0xdcd1 bytes) 2024-09-16T22:16:28.370 INFO:teuthology.orchestra.run.smithi031.stdout:787 write 0x555c76a thru 0x556b5bd (0xee54 bytes) 2024-09-16T22:16:28.380 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:28.381+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:28.380 INFO:teuthology.orchestra.run.smithi031.stdout:789 write 0x6eba89b thru 0x6ebf878 (0x4fde bytes) 2024-09-16T22:16:28.383 INFO:teuthology.orchestra.run.smithi031.stdout:791 read 0x48aafa5 thru 0x48aceb6 (0x1f12 bytes) 2024-09-16T22:16:28.383 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.383 INFO:teuthology.orchestra.run.smithi031.stdout:793 write 0x53d0535 thru 0x53da67d (0xa149 bytes) 2024-09-16T22:16:28.386 INFO:teuthology.orchestra.run.smithi031.stdout:796 write 0x5a8f8d9 thru 0x5a9a915 (0xb03d bytes) 2024-09-16T22:16:28.394 INFO:teuthology.orchestra.run.smithi031.stdout:797 writesame 0x8997ab1 thru 0x8997c3c data_size 0xc(0x18c bytes) 2024-09-16T22:16:28.394 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.394 INFO:teuthology.orchestra.run.smithi031.stdout:800 read 0x23d4177 thru 0x23e1714 (0xd59e bytes) 2024-09-16T22:16:28.395 INFO:teuthology.orchestra.run.smithi031.stdout:801 read 0x6c32d82 thru 0x6c3aab2 (0x7d31 bytes) 2024-09-16T22:16:28.395 INFO:teuthology.orchestra.run.smithi031.stdout:803 punch from 0x3eeb2f0 to 0x3eec70a, (0x141a bytes) 2024-09-16T22:16:28.395 INFO:teuthology.orchestra.run.smithi031.stdout:805 writesame 0x696379b thru 0x696390f data_size 0x1(0x175 bytes) 2024-09-16T22:16:28.397 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.397 INFO:teuthology.orchestra.run.smithi031.stdout:810 read 0x85d3485 thru 0x85e29e3 (0xf55f bytes) 2024-09-16T22:16:28.404 INFO:teuthology.orchestra.run.smithi031.stdout:811 write 0xa09a759 thru 0xa09b635 (0xedd bytes) 2024-09-16T22:16:28.407 INFO:teuthology.orchestra.run.smithi031.stdout:812 read 0x42b2d3 thru 0x42cbaa (0x18d8 bytes) 2024-09-16T22:16:28.407 INFO:teuthology.orchestra.run.smithi031.stdout:813 write 0x4057abe thru 0x40599b9 (0x1efc bytes) 2024-09-16T22:16:28.408 INFO:teuthology.orchestra.run.smithi031.stdout:814 punch from 0x263e001 to 0x264b813, (0xd812 bytes) 2024-09-16T22:16:28.408 INFO:teuthology.orchestra.run.smithi031.stdout:815 read 0x5eae14a thru 0x5eb5c36 (0x7aed bytes) 2024-09-16T22:16:28.408 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.408 INFO:teuthology.orchestra.run.smithi031.stdout:817 read 0x1f8c5ae thru 0x1f99ede (0xd931 bytes) 2024-09-16T22:16:28.416 INFO:teuthology.orchestra.run.smithi031.stdout:818 write 0xc60a070 thru 0xc60ddcd (0x3d5e bytes) 2024-09-16T22:16:28.419 INFO:teuthology.orchestra.run.smithi031.stdout:819 writesame 0x6474c10 thru 0x6479e48 data_size 0x1f(0x5239 bytes) 2024-09-16T22:16:28.422 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.422 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:28.432 INFO:teuthology.orchestra.run.smithi031.stdout:821 trunc from 0xc60ddce to 0x35d59be 2024-09-16T22:16:28.444 INFO:teuthology.orchestra.run.smithi031.stdout:822 write 0x4e302f thru 0x4e662c (0x35fe bytes) 2024-09-16T22:16:28.447 INFO:teuthology.orchestra.run.smithi031.stdout:824 read 0x254a964 thru 0x255a428 (0xfac5 bytes) 2024-09-16T22:16:28.448 INFO:teuthology.orchestra.run.smithi031.stdout:825 punch from 0x8fd2e to 0x98ec9, (0x919b bytes) 2024-09-16T22:16:28.448 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.448 INFO:teuthology.orchestra.run.smithi031.stdout:828 trunc from 0x35d59be to 0x30e7ffa 2024-09-16T22:16:28.455 INFO:teuthology.orchestra.run.smithi031.stdout:829 punch from 0x20f2902 to 0x20f861e, (0x5d1c bytes) 2024-09-16T22:16:28.470 INFO:teuthology.orchestra.run.smithi031.stdout:831 writesame 0x99ab976 thru 0x99abe12 data_size 0x1(0x49d bytes) 2024-09-16T22:16:28.471 INFO:teuthology.orchestra.run.smithi031.stdout:832 punch from 0x47a3a97 to 0x47a3d87, (0x2f0 bytes) 2024-09-16T22:16:28.471 INFO:teuthology.orchestra.run.smithi031.stdout:834 write 0x7902acf thru 0x7908847 (0x5d79 bytes) 2024-09-16T22:16:28.473 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.473 INFO:teuthology.orchestra.run.smithi031.stdout:836 write 0x29bc732 thru 0x29c22e5 (0x5bb4 bytes) 2024-09-16T22:16:28.475 INFO:teuthology.orchestra.run.smithi031.stdout:837 writesame 0x58ed7a4 thru 0x58f30a5 data_size 0x1(0x5902 bytes) 2024-09-16T22:16:28.480 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:28.482+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:28.488 INFO:teuthology.orchestra.run.smithi031.stdout:838 punch from 0x6afa414 to 0x6b05d92, (0xb97e bytes) 2024-09-16T22:16:28.488 INFO:teuthology.orchestra.run.smithi031.stdout:839 write 0x98c69e8 thru 0x98d056c (0x9b85 bytes) 2024-09-16T22:16:28.492 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.501 INFO:teuthology.orchestra.run.smithi031.stdout:841 write 0xb3992a2 thru 0xb39d39d (0x40fc bytes) 2024-09-16T22:16:28.505 INFO:teuthology.orchestra.run.smithi031.stdout:842 write 0x562fbc5 thru 0x5630a40 (0xe7c bytes) 2024-09-16T22:16:28.508 INFO:teuthology.orchestra.run.smithi031.stdout:843 write 0x591d89c thru 0x592a9c6 (0xd12b bytes) 2024-09-16T22:16:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:844 read 0x4fbda0b thru 0x4fbfd27 (0x231d bytes) 2024-09-16T22:16:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:845 writesame 0xa92110a thru 0xa925031 data_size 0x8(0x3f28 bytes) 2024-09-16T22:16:28.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.515 INFO:teuthology.orchestra.run.smithi031.stdout:847 write 0x8dd14ac thru 0x8de1352 (0xfea7 bytes) 2024-09-16T22:16:28.518 INFO:teuthology.orchestra.run.smithi031.stdout:848 write 0x8e5ffaa thru 0x8e63a47 (0x3a9e bytes) 2024-09-16T22:16:28.519 INFO:teuthology.orchestra.run.smithi031.stdout:849 read 0x7159dc5 thru 0x715b3de (0x161a bytes) 2024-09-16T22:16:28.519 INFO:teuthology.orchestra.run.smithi031.stdout:850 read 0x2b7dd0f thru 0x2b8a16b (0xc45d bytes) 2024-09-16T22:16:28.523 INFO:teuthology.orchestra.run.smithi031.stdout:851 trunc from 0xb39d39e to 0x81c356e 2024-09-16T22:16:28.540 INFO:teuthology.orchestra.run.smithi031.stdout:854 trunc from 0x81c356e to 0x2a77455 2024-09-16T22:16:28.541 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.541 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:28.553 INFO:teuthology.orchestra.run.smithi031.stdout:855 read 0x217ab74 thru 0x2185fd5 (0xb462 bytes) 2024-09-16T22:16:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:856 writesame 0xc2794b5 thru 0xc284909 data_size 0x7(0xb455 bytes) 2024-09-16T22:16:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:857 read 0x2ffd68 thru 0x30477c (0x4a15 bytes) 2024-09-16T22:16:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:860 flatten 2024-09-16T22:16:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:862 write 0x7ce0c9f thru 0x7ce678e (0x5af0 bytes) 2024-09-16T22:16:28.581 INFO:teuthology.orchestra.run.smithi031.stdout:864 writesame 0x43d8ff3 thru 0x43e7b0f data_size 0x1(0xeb1d bytes) 2024-09-16T22:16:28.581 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:28.583+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:28.612 INFO:teuthology.orchestra.run.smithi031.stdout:865 read 0x8d6f453 thru 0x8d730f4 (0x3ca2 bytes) 2024-09-16T22:16:28.621 INFO:teuthology.orchestra.run.smithi031.stdout:866 trunc from 0xc28490a to 0x3ee9cf1 2024-09-16T22:16:28.633 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:28.633 INFO:teuthology.orchestra.run.smithi031.stdout:868 write 0x35703df thru 0x3575487 (0x50a9 bytes) 2024-09-16T22:16:28.635 INFO:teuthology.orchestra.run.smithi031.stdout:869 clone 6 order 22 su 262144 sc 12 2024-09-16T22:16:28.659 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.659 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:28.682 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:28.684+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:28.775 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.775 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.783 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:28.784+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:28.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:28.883 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:28.885+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:28.931 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:28.931 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:28.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:29.024 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:29.026+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:29.086 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:29.087+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:29.088 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.088 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:29.187 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:29.188+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:29.189 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:16:29.253 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.254 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:29.300 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T22:16:29.300 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:29.302+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:29.301 INFO:teuthology.orchestra.run.smithi031.stdout:870 write 0x351ead3 thru 0x352197d (0x2eab bytes) 2024-09-16T22:16:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:871 writesame 0xd529265 thru 0xd52bcd9 data_size 0x1(0x2a75 bytes) 2024-09-16T22:16:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:872 flatten 2024-09-16T22:16:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:873 trunc from 0xd52bcda to 0x9308183 2024-09-16T22:16:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:876 read 0x922dde6 thru 0x923879e (0xa9b9 bytes) 2024-09-16T22:16:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:877 write 0x3314e28 thru 0x3320627 (0xb800 bytes) 2024-09-16T22:16:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:878 trunc from 0x9308183 to 0x8b9c60a 2024-09-16T22:16:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:879 writesame 0x535da thru 0x5de6b data_size 0x1(0xa892 bytes) 2024-09-16T22:16:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:881 write 0x7445f7c thru 0x7452519 (0xc59e bytes) 2024-09-16T22:16:29.389 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:29.391+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:883 write 0x3f5e68b thru 0x3f67645 (0x8fbb bytes) 2024-09-16T22:16:29.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:884 trunc from 0x8b9c60a to 0x59cbf2e 2024-09-16T22:16:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:885 writesame 0x89f72e3 thru 0x89faadd data_size 0x11(0x37fb bytes) 2024-09-16T22:16:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:886 writesame 0xca5982f thru 0xca5fa69 data_size 0x1(0x623b bytes) 2024-09-16T22:16:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:887 write 0xd9eba0c thru 0xd9fb7eb (0xfde0 bytes) 2024-09-16T22:16:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:888 writesame 0x5a0a7fc thru 0x5a109bc data_size 0x4d(0x61c1 bytes) 2024-09-16T22:16:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:889 trunc from 0xd9fb7ec to 0x59c57f9 2024-09-16T22:16:29.463 INFO:teuthology.orchestra.run.smithi031.stdout:891 write 0x7c53713 thru 0x7c61289 (0xdb77 bytes) 2024-09-16T22:16:29.469 INFO:teuthology.orchestra.run.smithi031.stdout:893 trunc from 0x7c6128a to 0xa9679ba 2024-09-16T22:16:29.474 INFO:teuthology.orchestra.run.smithi031.stdout:894 read 0xa742942 thru 0xa743533 (0xbf2 bytes) 2024-09-16T22:16:29.474 INFO:teuthology.orchestra.run.smithi031.stdout:895 read 0x443a7f2 thru 0x44498df (0xf0ee bytes) 2024-09-16T22:16:29.474 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.475 INFO:teuthology.orchestra.run.smithi031.stdout:897 write 0x84f4f3b thru 0x84fb469 (0x652f bytes) 2024-09-16T22:16:29.477 INFO:teuthology.orchestra.run.smithi031.stdout:898 read 0x7d29abb thru 0x7d32ab6 (0x8ffc bytes) 2024-09-16T22:16:29.477 INFO:teuthology.orchestra.run.smithi031.stdout:899 write 0x8e620ac thru 0x8e6a57c (0x84d1 bytes) 2024-09-16T22:16:29.479 INFO:teuthology.orchestra.run.smithi031.stdout:901 read 0x9e89ab4 thru 0x9e8f7d3 (0x5d20 bytes) 2024-09-16T22:16:29.479 INFO:teuthology.orchestra.run.smithi031.stdout:904 read 0x88a647f thru 0x88b5e74 (0xf9f6 bytes) 2024-09-16T22:16:29.485 INFO:teuthology.orchestra.run.smithi031.stdout:905 trunc from 0xa9679ba to 0x59a0878 2024-09-16T22:16:29.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:29.491+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:29.495 INFO:teuthology.orchestra.run.smithi031.stdout:906 punch from 0xfc1df3 to 0xfc9369, (0x7576 bytes) 2024-09-16T22:16:29.495 INFO:teuthology.orchestra.run.smithi031.stdout:907 read 0x56cc2d1 thru 0x56db798 (0xf4c8 bytes) 2024-09-16T22:16:29.495 INFO:teuthology.orchestra.run.smithi031.stdout:908 punch from 0x36e0bce to 0x36e505b, (0x448d bytes) 2024-09-16T22:16:29.496 INFO:teuthology.orchestra.run.smithi031.stdout:913 writesame 0x176615e thru 0x176e26e data_size 0x25(0x8111 bytes) 2024-09-16T22:16:29.502 INFO:teuthology.orchestra.run.smithi031.stdout:916 trunc from 0x59a0878 to 0x28cc38c 2024-09-16T22:16:29.516 INFO:teuthology.orchestra.run.smithi031.stdout:920 read 0xab1cff thru 0xabe6c1 (0xc9c3 bytes) 2024-09-16T22:16:29.518 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.519 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:921 trunc from 0x28cc38c to 0x819b4ce 2024-09-16T22:16:29.528 INFO:teuthology.orchestra.run.smithi031.stdout:924 write 0x74d64f8 thru 0x74e20c2 (0xbbcb bytes) 2024-09-16T22:16:29.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:29.532 INFO:teuthology.orchestra.run.smithi031.stdout:925 read 0x70e98df thru 0x70eba97 (0x21b9 bytes) 2024-09-16T22:16:29.532 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.532 INFO:teuthology.orchestra.run.smithi031.stdout:928 read 0x114a29d thru 0x1156fd6 (0xcd3a bytes) 2024-09-16T22:16:29.532 INFO:teuthology.orchestra.run.smithi031.stdout:929 read 0x1a59d1e thru 0x1a5eaef (0x4dd2 bytes) 2024-09-16T22:16:29.541 INFO:teuthology.orchestra.run.smithi031.stdout:930 trunc from 0x819b4ce to 0x30eaaa 2024-09-16T22:16:29.570 INFO:teuthology.orchestra.run.smithi031.stdout:931 write 0xc19ded1 thru 0xc1a797a (0x9aaa bytes) 2024-09-16T22:16:29.573 INFO:teuthology.orchestra.run.smithi031.stdout:933 read 0x67a23a4 thru 0x67aa6e4 (0x8341 bytes) 2024-09-16T22:16:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:934 write 0x5e78536 thru 0x5e8824a (0xfd15 bytes) 2024-09-16T22:16:29.577 INFO:teuthology.orchestra.run.smithi031.stdout:935 write 0x56ebe88 thru 0x56f3209 (0x7382 bytes) 2024-09-16T22:16:29.581 INFO:teuthology.orchestra.run.smithi031.stdout:936 read 0xbe24737 thru 0xbe29e62 (0x572c bytes) 2024-09-16T22:16:29.581 INFO:teuthology.orchestra.run.smithi031.stdout:937 read 0x6bcfb4f thru 0x6bd6a75 (0x6f27 bytes) 2024-09-16T22:16:29.581 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.581 INFO:teuthology.orchestra.run.smithi031.stdout:939 write 0x772081d thru 0x77285f1 (0x7dd5 bytes) 2024-09-16T22:16:29.591 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:29.593+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:29.594 INFO:teuthology.orchestra.run.smithi031.stdout:941 trunc from 0xc1a797b to 0x28fb81f 2024-09-16T22:16:29.606 INFO:teuthology.orchestra.run.smithi031.stdout:942 read 0x9383c8 thru 0x93f330 (0x6f69 bytes) 2024-09-16T22:16:29.607 INFO:teuthology.orchestra.run.smithi031.stdout:946 writesame 0xbd9c23 thru 0xbe9028 data_size 0x5(0xf406 bytes) 2024-09-16T22:16:29.612 INFO:teuthology.orchestra.run.smithi031.stdout:948 punch from 0x41a2b4 to 0x424373, (0xa0bf bytes) 2024-09-16T22:16:29.626 INFO:teuthology.orchestra.run.smithi031.stdout:949 write 0x9bbbb10 thru 0x9bc87e9 (0xccda bytes) 2024-09-16T22:16:29.642 INFO:teuthology.orchestra.run.smithi031.stdout:950 write 0xe952b3e thru 0xe959a3d (0x6f00 bytes) 2024-09-16T22:16:29.645 INFO:teuthology.orchestra.run.smithi031.stdout:951 read 0xd06d2f2 thru 0xd071d9d (0x4aac bytes) 2024-09-16T22:16:29.646 INFO:teuthology.orchestra.run.smithi031.stdout:952 write 0xe683341 thru 0xe68de20 (0xaae0 bytes) 2024-09-16T22:16:29.648 INFO:teuthology.orchestra.run.smithi031.stdout:953 punch from 0x5317011 to 0x531c108, (0x50f7 bytes) 2024-09-16T22:16:29.654 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.654 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.662 INFO:teuthology.orchestra.run.smithi031.stdout:954 trunc from 0xe959a3e to 0x28a7c5e 2024-09-16T22:16:29.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:29.694 INFO:teuthology.orchestra.run.smithi031.stdout:955 write 0xe052055 thru 0xe061ef6 (0xfea2 bytes) 2024-09-16T22:16:29.694 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:29.695+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:29.710 INFO:teuthology.orchestra.run.smithi031.stdout:957 trunc from 0xe061ef7 to 0x2f4c4a3 2024-09-16T22:16:29.734 INFO:teuthology.orchestra.run.smithi031.stdout:960 write 0x9d11f19 thru 0x9d21128 (0xf210 bytes) 2024-09-16T22:16:29.740 INFO:teuthology.orchestra.run.smithi031.stdout:963 trunc from 0x9d21129 to 0xbd69d53 2024-09-16T22:16:29.753 INFO:teuthology.orchestra.run.smithi031.stdout:965 writesame 0xd224bce thru 0xd229d43 data_size 0x1(0x5176 bytes) 2024-09-16T22:16:29.753 INFO:teuthology.orchestra.run.smithi031.stdout:966 read 0x253d3c3 thru 0x25450dc (0x7d1a bytes) 2024-09-16T22:16:29.753 INFO:teuthology.orchestra.run.smithi031.stdout:967 punch from 0x6316e49 to 0x631fcb4, (0x8e6b bytes) 2024-09-16T22:16:29.753 INFO:teuthology.orchestra.run.smithi031.stdout:968 write 0x78eeef6 thru 0x78f4183 (0x528e bytes) 2024-09-16T22:16:29.755 INFO:teuthology.orchestra.run.smithi031.stdout:969 write 0x8ab34f9 thru 0x8ac2f28 (0xfa30 bytes) 2024-09-16T22:16:29.758 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.758 INFO:teuthology.orchestra.run.smithi031.stdout:972 writesame 0x377b6f2 thru 0x37802ce data_size 0x1(0x4bdd bytes) 2024-09-16T22:16:29.769 INFO:teuthology.orchestra.run.smithi031.stdout:974 write 0x8d77bc3 thru 0x8d811f8 (0x9636 bytes) 2024-09-16T22:16:29.772 INFO:teuthology.orchestra.run.smithi031.stdout:975 read 0x2f57891 thru 0x2f5c64f (0x4dbf bytes) 2024-09-16T22:16:29.772 INFO:teuthology.orchestra.run.smithi031.stdout:977 write 0x7d9294b thru 0x7d981f8 (0x58ae bytes) 2024-09-16T22:16:29.776 INFO:teuthology.orchestra.run.smithi031.stdout:979 write 0x93752be thru 0x937c1b2 (0x6ef5 bytes) 2024-09-16T22:16:29.779 INFO:teuthology.orchestra.run.smithi031.stdout:980 read 0x496e7cc thru 0x496ebb8 (0x3ed bytes) 2024-09-16T22:16:29.779 INFO:teuthology.orchestra.run.smithi031.stdout:981 writesame 0x298ca63 thru 0x2999596 data_size 0x66(0xcb34 bytes) 2024-09-16T22:16:29.780 INFO:teuthology.orchestra.run.smithi031.stdout:983 write 0xc2c99f7 thru 0xc2c9afb (0x105 bytes) 2024-09-16T22:16:29.784 INFO:teuthology.orchestra.run.smithi031.stdout:984 trunc from 0xd229d44 to 0xbfcd599 2024-09-16T22:16:29.792 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:29.794+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:29.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.794 INFO:teuthology.orchestra.run.smithi031.stdout:988 read 0x644f9fb thru 0x6451f0a (0x2510 bytes) 2024-09-16T22:16:29.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.794 INFO:teuthology.orchestra.run.smithi031.stdout:990 writesame 0x8e067fb thru 0x8e0b3de data_size 0x3(0x4be4 bytes) 2024-09-16T22:16:29.799 INFO:teuthology.orchestra.run.smithi031.stdout:991 punch from 0xa125f27 to 0xa12afc1, (0x509a bytes) 2024-09-16T22:16:29.799 INFO:teuthology.orchestra.run.smithi031.stdout:992 writesame 0x2d70f3a thru 0x2d7d76e data_size 0x1(0xc835 bytes) 2024-09-16T22:16:29.815 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.815 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.824 INFO:teuthology.orchestra.run.smithi031.stdout:995 punch from 0x79be45d to 0x79bf32e, (0xed1 bytes) 2024-09-16T22:16:29.824 INFO:teuthology.orchestra.run.smithi031.stdout:996 write 0x21d7dcc thru 0x21d9d2d (0x1f62 bytes) 2024-09-16T22:16:29.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:29.828 INFO:teuthology.orchestra.run.smithi031.stdout:997 read 0x3ba027b thru 0x3ba10e8 (0xe6e bytes) 2024-09-16T22:16:29.828 INFO:teuthology.orchestra.run.smithi031.stdout:998 read 0x26c5633 thru 0x26d1b97 (0xc565 bytes) 2024-09-16T22:16:29.828 INFO:teuthology.orchestra.run.smithi031.stdout:999 read 0x45d0ee2 thru 0x45d941d (0x853c bytes) 2024-09-16T22:16:29.834 INFO:teuthology.orchestra.run.smithi031.stdout:1000 write 0xcf1a70f thru 0xcf1f7ba (0x50ac bytes) 2024-09-16T22:16:29.846 INFO:teuthology.orchestra.run.smithi031.stdout:1001 write 0xe38f4a3 thru 0xe396508 (0x7066 bytes) 2024-09-16T22:16:29.848 INFO:teuthology.orchestra.run.smithi031.stdout:1002 read 0xac18c83 thru 0xac1dc17 (0x4f95 bytes) 2024-09-16T22:16:29.848 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:29.849 INFO:teuthology.orchestra.run.smithi031.stdout:1004 writesame 0x9c9b177 thru 0x9ca2b17 data_size 0x61(0x79a1 bytes) 2024-09-16T22:16:29.852 INFO:teuthology.orchestra.run.smithi031.stdout:1006 read 0x73c3722 thru 0x73d2cfa (0xf5d9 bytes) 2024-09-16T22:16:29.852 INFO:teuthology.orchestra.run.smithi031.stdout:1008 punch from 0x5f5e872 to 0x5f64beb, (0x6379 bytes) 2024-09-16T22:16:29.852 INFO:teuthology.orchestra.run.smithi031.stdout:1009 read 0x19e1532 thru 0x19e45bb (0x308a bytes) 2024-09-16T22:16:29.852 INFO:teuthology.orchestra.run.smithi031.stdout:1011 read 0x3ddca9c thru 0x3de97b6 (0xcd1b bytes) 2024-09-16T22:16:29.852 INFO:teuthology.orchestra.run.smithi031.stdout:1012 writesame 0x6bac55c thru 0x6bb45d6 data_size 0x1(0x807b bytes) 2024-09-16T22:16:29.869 INFO:teuthology.orchestra.run.smithi031.stdout:1013 writesame 0xa035307 thru 0xa0392cc data_size 0x9(0x3fc6 bytes) 2024-09-16T22:16:29.871 INFO:teuthology.orchestra.run.smithi031.stdout:1014 write 0x22715a thru 0x231b83 (0xaa2a bytes) 2024-09-16T22:16:29.873 INFO:teuthology.orchestra.run.smithi031.stdout:1017 write 0x2a03707 thru 0x2a0e5c3 (0xaebd bytes) 2024-09-16T22:16:29.874 INFO:teuthology.orchestra.run.smithi031.stdout:1018 read 0xcd49ca0 thru 0xcd4ea84 (0x4de5 bytes) 2024-09-16T22:16:29.875 INFO:teuthology.orchestra.run.smithi031.stdout:1019 write 0x384129c thru 0x384fc44 (0xe9a9 bytes) 2024-09-16T22:16:29.877 INFO:teuthology.orchestra.run.smithi031.stdout:1020 punch from 0x41ec0fa to 0x41fbf22, (0xfe28 bytes) 2024-09-16T22:16:29.878 INFO:teuthology.orchestra.run.smithi031.stdout:1021 punch from 0x1253f27 to 0x1261b2a, (0xdc03 bytes) 2024-09-16T22:16:29.886 INFO:teuthology.orchestra.run.smithi031.stdout:1022 trunc from 0xe396509 to 0x6387e26 2024-09-16T22:16:29.893 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:29.895+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:29.897 INFO:teuthology.orchestra.run.smithi031.stdout:1023 read 0x4e53e33 thru 0x4e5ec64 (0xae32 bytes) 2024-09-16T22:16:29.903 INFO:teuthology.orchestra.run.smithi031.stdout:1024 trunc from 0x6387e26 to 0x16c8d70 2024-09-16T22:16:29.917 INFO:teuthology.orchestra.run.smithi031.stdout:1025 punch from 0x1431b91 to 0x1438a17, (0x6e86 bytes) 2024-09-16T22:16:29.918 INFO:teuthology.orchestra.run.smithi031.stdout:1026 trunc from 0x16c8d70 to 0x236b070 2024-09-16T22:16:29.937 INFO:teuthology.orchestra.run.smithi031.stdout:1027 trunc from 0x236b070 to 0xeb0ba2d 2024-09-16T22:16:29.942 INFO:teuthology.orchestra.run.smithi031.stdout:1029 read 0x99da5ad thru 0x99e9a14 (0xf468 bytes) 2024-09-16T22:16:29.948 INFO:teuthology.orchestra.run.smithi031.stdout:1030 trunc from 0xeb0ba2d to 0xa317a3a 2024-09-16T22:16:29.948 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:29.948 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:1031 punch from 0x93a3c81 to 0x93ad541, (0x98c0 bytes) 2024-09-16T22:16:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:1032 punch from 0x97e37a3 to 0x97e687d, (0x30da bytes) 2024-09-16T22:16:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:1033 punch from 0x9b5f87a to 0x9b6a3b7, (0xab3d bytes) 2024-09-16T22:16:29.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:29.964 INFO:teuthology.orchestra.run.smithi031.stdout:1034 writesame 0xe4ceca6 thru 0xe4ced36 data_size 0x1(0x91 bytes) 2024-09-16T22:16:29.965 INFO:teuthology.orchestra.run.smithi031.stdout:1035 writesame 0x46ad572 thru 0x46b4aee data_size 0x1(0x757d bytes) 2024-09-16T22:16:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:1036 read 0x1bebdb1 thru 0x1bf52dd (0x952d bytes) 2024-09-16T22:16:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:1037 write 0x24b4c89 thru 0x24b826d (0x35e5 bytes) 2024-09-16T22:16:29.984 INFO:teuthology.orchestra.run.smithi031.stdout:1041 read 0x87d12ab thru 0x87d6962 (0x56b8 bytes) 2024-09-16T22:16:29.984 INFO:teuthology.orchestra.run.smithi031.stdout:1042 read 0x2b93be8 thru 0x2ba2d69 (0xf182 bytes) 2024-09-16T22:16:29.984 INFO:teuthology.orchestra.run.smithi031.stdout:1044 write 0x2fd5f8e thru 0x2fe5406 (0xf479 bytes) 2024-09-16T22:16:29.995 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:29.997+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:29.998 INFO:teuthology.orchestra.run.smithi031.stdout:1045 trunc from 0xe4ced37 to 0x3753af7 2024-09-16T22:16:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:1046 writesame 0x6499372 thru 0x64a13a6 data_size 0x1(0x8035 bytes) 2024-09-16T22:16:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:1049 read 0x33ecd60 thru 0x33fa12f (0xd3d0 bytes) 2024-09-16T22:16:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:1051 read 0x179f0f7 thru 0x17aaad9 (0xb9e3 bytes) 2024-09-16T22:16:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:1053 writesame 0x56e951f thru 0x56ea87b data_size 0x1(0x135d bytes) 2024-09-16T22:16:30.018 INFO:teuthology.orchestra.run.smithi031.stdout:1055 read 0x38da97c thru 0x38e5f46 (0xb5cb bytes) 2024-09-16T22:16:30.018 INFO:teuthology.orchestra.run.smithi031.stdout:1057 read 0x4c439b0 thru 0x4c51e4e (0xe49f bytes) 2024-09-16T22:16:30.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:30.033 INFO:teuthology.orchestra.run.smithi031.stdout:1058 writesame 0xdec247c thru 0xded1ba4 data_size 0x17(0xf729 bytes) 2024-09-16T22:16:30.047 INFO:teuthology.orchestra.run.smithi031.stdout:1059 trunc from 0xded1ba5 to 0x192e20a 2024-09-16T22:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:1061 writesame 0x19e7596 thru 0x19f57a7 data_size 0x13(0xe212 bytes) 2024-09-16T22:16:30.084 INFO:teuthology.orchestra.run.smithi031.stdout:1062 write 0xa031c8d thru 0xa037894 (0x5c08 bytes) 2024-09-16T22:16:30.085 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:30.085 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:30.088 INFO:teuthology.orchestra.run.smithi031.stdout:1063 punch from 0x6e42011 to 0x6e45aeb, (0x3ada bytes) 2024-09-16T22:16:30.088 INFO:teuthology.orchestra.run.smithi031.stdout:1065 write 0x96a492e thru 0x96a911c (0x47ef bytes) 2024-09-16T22:16:30.096 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:30.098+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:30.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:30.101 INFO:teuthology.orchestra.run.smithi031.stdout:1066 write 0xe577dd9 thru 0xe579f9d (0x21c5 bytes) 2024-09-16T22:16:30.104 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:30.119 INFO:teuthology.orchestra.run.smithi031.stdout:1069 trunc from 0xe579f9e to 0x6bcd86 2024-09-16T22:16:30.132 INFO:teuthology.orchestra.run.smithi031.stdout:1070 punch from 0x6112e1 to 0x61497a, (0x3699 bytes) 2024-09-16T22:16:30.132 INFO:teuthology.orchestra.run.smithi031.stdout:1071 read 0x14439b thru 0x14604c (0x1cb2 bytes) 2024-09-16T22:16:30.133 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:30.143 INFO:teuthology.orchestra.run.smithi031.stdout:1075 write 0x58812aa thru 0x58903aa (0xf101 bytes) 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:1076 read 0xe8d45a thru 0xe8e5d0 (0x1177 bytes) 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:1077 read 0x56abb51 thru 0x56b85f5 (0xcaa5 bytes) 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:1079 punch from 0x3c06710 to 0x3c09e77, (0x3767 bytes) 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:1080 read 0x57201bd thru 0x572eacb (0xe90f bytes) 2024-09-16T22:16:30.149 INFO:teuthology.orchestra.run.smithi031.stdout:1081 read 0x4105372 thru 0x410dbb4 (0x8843 bytes) 2024-09-16T22:16:30.159 INFO:teuthology.orchestra.run.smithi031.stdout:1082 write 0x9bd7d6e thru 0x9bdb403 (0x3696 bytes) 2024-09-16T22:16:30.162 INFO:teuthology.orchestra.run.smithi031.stdout:1083 read 0x5765a89 thru 0x576c62e (0x6ba6 bytes) 2024-09-16T22:16:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:1084 writesame 0xc9d650e thru 0xc9e607d data_size 0x95(0xfb70 bytes) 2024-09-16T22:16:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:1085 punch from 0x57f91a2 to 0x580010e, (0x6f6c bytes) 2024-09-16T22:16:30.172 INFO:teuthology.orchestra.run.smithi031.stdout:1086 read 0x94ae46 thru 0x94e9bd (0x3b78 bytes) 2024-09-16T22:16:30.172 INFO:teuthology.orchestra.run.smithi031.stdout:1087 write 0x37d72bb thru 0x37d9be3 (0x2929 bytes) 2024-09-16T22:16:30.174 INFO:teuthology.orchestra.run.smithi031.stdout:1089 punch from 0x230de5c to 0x2319837, (0xb9db bytes) 2024-09-16T22:16:30.174 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:30.176 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:30.177 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:30.189 INFO:teuthology.orchestra.run.smithi031.stdout:1092 trunc from 0xc9e607e to 0x10496c9 2024-09-16T22:16:30.197 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:30.197 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:30.198+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:30.197 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:30.203 INFO:teuthology.orchestra.run.smithi031.stdout:1093 clone 7 order 25 su 16777216 sc 15 2024-09-16T22:16:30.214 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:30.230 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:30.235 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:30.235 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:30.235 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:30.235 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:30.235 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:30.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:30.248 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:30.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:30.265 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:30.282 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:30.282 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:30.297 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:30.299+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:30.299 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.930342487000001, 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:30.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:30.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1058 3.1d 3:bd9a1188:::rbd_data.11cd70c2eac3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5214116020000006, 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063284730000000003, 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1058 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836421+0000", 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836422+0000", 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836426+0000", 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836435+0000", 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0610000000000002e-06 2024-09-16T22:16:30.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836439+0000", 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4619999999999998e-06 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836483+0000", 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-05 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.836536+0000", 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.837607+0000", 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010716250000000001 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842680+0000", 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050721500000000001 2024-09-16T22:16:30.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842692+0000", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1960000000000001e-05 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842721+0000", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9521000000000001e-05 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.842750+0000", 2024-09-16T22:16:30.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8483000000000002e-05 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1403 3.d 3:b26fa55d:::rbd_data.11cd70c2eac3.0000000000000005:head [stat,truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1786443769999995, 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060892699999999999, 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1403 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179188+0000", 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179189+0000", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179192+0000", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.157e-06 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179447+0000", 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025534700000000001 2024-09-16T22:16:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179457+0000", 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179525+0000", 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599000000000001e-05 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.179662+0000", 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013697700000000001 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.180004+0000", 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000341916 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185068+0000", 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050640370000000004 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185199+0000", 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013083600000000001 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185243+0000", 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4557e-05 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.185278+0000", 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4518999999999999e-05 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1882 3.9 3:923ad63a:::rbd_data.11cd70c2eac3.0000000000000002:head [truncate 1441792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.562769211, 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006568979, 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1882 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795064+0000", 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795065+0000", 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.221e-06 2024-09-16T22:16:30.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795070+0000", 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795085+0000", 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4696e-05 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795092+0000", 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7330000000000003e-06 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795165+0000", 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2428999999999994e-05 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795259+0000", 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4246000000000005e-05 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.795379+0000", 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011969899999999999 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.796874+0000", 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001495637 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801529+0000", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046545900000000001 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801587+0000", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8588999999999998e-05 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.801632+0000", 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5040000000000002e-05 2024-09-16T22:16:30.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1963 3.14 3:281b8854:::rbd_data.11cd70c2eac3.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.4548195330000002, 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01015051, 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1963 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903013+0000", 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903014+0000", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903018+0000", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0010000000000003e-06 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903028+0000", 2024-09-16T22:16:30.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903034+0000", 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4670000000000001e-06 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903076+0000", 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1900000000000002e-05 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.903123+0000", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7178000000000001e-05 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.904508+0000", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013844110000000001 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913089+0000", 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085809930000000003 2024-09-16T22:16:30.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913100+0000", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0868e-05 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913132+0000", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2632999999999997e-05 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.913164+0000", 2024-09-16T22:16:30.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1316000000000003e-05 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9872 3.11 3:89d9ffed:::rbd_data.11cdc482f8dc.00000000000000b3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5501065790000004, 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082067210000000002, 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9872 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807726+0000", 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807727+0000", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5000000000000001e-07 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807731+0000", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807740+0000", 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9320000000000002e-06 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807746+0000", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1720000000000004e-06 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807804+0000", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8088999999999999e-05 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.807873+0000", 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8621999999999995e-05 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.808064+0000", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019087999999999999 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.810113+0000", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020487769999999999 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815829+0000", 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057160559999999997 2024-09-16T22:16:30.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815885+0000", 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489000000000001e-05 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.815933+0000", 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7728000000000001e-05 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9883 3.d 3:b5c08af5:::rbd_data.11cdc482f8dc.00000000000000bc:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.544233856, 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063131180000000004, 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9883 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813599+0000", 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813600+0000", 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6899999999999997e-07 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813602+0000", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9289999999999998e-06 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813878+0000", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027559200000000001 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.813888+0000", 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5349999999999993e-06 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.814099+0000", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000211177 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.814223+0000", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012391100000000001 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.819912+0000", 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056893050000000004 2024-09-16T22:16:30.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9899 3.d 3:b5c08af5:::rbd_data.11cdc482f8dc.00000000000000bc:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5370344149999999, 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066083340000000004, 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9899 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820798+0000", 2024-09-16T22:16:30.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820799+0000", 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4600000000000003e-07 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820805+0000", 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6359999999999999e-06 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820817+0000", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2554e-05 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.820824+0000", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7859999999999999e-06 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821023+0000", 2024-09-16T22:16:30.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019892600000000001 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821102+0000", 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8920999999999999e-05 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.821307+0000", 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020460899999999999 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.822279+0000", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00097247000000000002 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827317+0000", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050373709999999997 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827372+0000", 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4978000000000001e-05 2024-09-16T22:16:30.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.827407+0000", 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5136999999999999e-05 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:9907 3.a 3:55fa9e5b:::rbd_data.11cdc482f8dc.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:30.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5328849629999999, 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086142360000000008, 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9907 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:30.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824948+0000", 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824949+0000", 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0400000000000005e-07 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824953+0000", 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.873e-06 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824963+0000", 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0061000000000001e-05 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.824967+0000", 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825021+0000", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067999999999998e-05 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825109+0000", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7311000000000005e-05 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.825307+0000", 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000198481 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.829904+0000", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045966729999999999 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833459+0000", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035554599999999999 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833521+0000", 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1753000000000003e-05 2024-09-16T22:16:30.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.833562+0000", 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1017999999999999e-05 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13070 3.11 3:8a7471e7:::rbd_data.11cdc482f8dc.0000000000000132:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2673838140000004, 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073813500000000001, 2024-09-16T22:16:30.411 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13070 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090449+0000", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090451+0000", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.922e-06 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090458+0000", 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6290000000000001e-06 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090484+0000", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6038e-05 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090497+0000", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2371999999999999e-05 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090565+0000", 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7861999999999995e-05 2024-09-16T22:16:30.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.090690+0000", 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012561399999999999 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.097830+0000", 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071399130000000003 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13330 3.13 3:cb153b9d:::rbd_data.11cdc482f8dc.0000000000000172:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2124801520000004, 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091642049999999999, 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13330 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145353+0000", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145354+0000", 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5900000000000005e-07 2024-09-16T22:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145359+0000", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145370+0000", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1107000000000001e-05 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145375+0000", 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:16:30.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145428+0000", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3375000000000003e-05 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145511+0000", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2578999999999997e-05 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145703+0000", 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019194200000000001 2024-09-16T22:16:30.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150012+0000", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043096810000000001 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154427+0000", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004414935 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154483+0000", 2024-09-16T22:16:30.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5333e-05 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154517+0000", 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4227000000000003e-05 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13335 3.14 3:2c852cac:::rbd_data.11cdc482f8dc.000000000000017c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2103652699999996, 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067530489999999997, 2024-09-16T22:16:30.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13335 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147468+0000", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0079999999999999e-06 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147474+0000", 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147490+0000", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6348999999999999e-05 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147498+0000", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6459999999999993e-06 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147560+0000", 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999995e-05 2024-09-16T22:16:30.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147657+0000", 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7127999999999996e-05 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154220+0000", 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065637960000000002 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13337 3.d 3:b6885ab0:::rbd_data.11cdc482f8dc.0000000000000176:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2072284670000002, 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066670519999999997, 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13337 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150604+0000", 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150605+0000", 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0999999999999998e-07 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150609+0000", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9600000000000002e-06 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150618+0000", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4690000000000003e-06 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150626+0000", 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0630000000000006e-06 2024-09-16T22:16:30.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150678+0000", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1576000000000002e-05 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150751+0000", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2941999999999994e-05 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150950+0000", 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019875399999999999 2024-09-16T22:16:30.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157112+0000", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061624089999999998 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157190+0000", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7756000000000006e-05 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157236+0000", 2024-09-16T22:16:30.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157271+0000", 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5587000000000003e-05 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13338 3.11 3:8d0fdcbe:::rbd_data.11cdc482f8dc.000000000000017a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2071834350000001, 2024-09-16T22:16:30.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063968640000000004, 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13338 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150649+0000", 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150650+0000", 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3499999999999996e-07 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150653+0000", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1429999999999998e-06 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150661+0000", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8220000000000007e-06 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150665+0000", 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3440000000000003e-06 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150714+0000", 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8483999999999999e-05 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150791+0000", 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7576000000000004e-05 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:30.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.150995+0000", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020399700000000001 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154972+0000", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039767350000000003 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.156963+0000", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019913079999999998 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157011+0000", 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7568999999999998e-05 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.157046+0000", 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5351000000000001e-05 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13389 3.11 3:89f76da2:::rbd_data.11cdc482f8dc.0000000000000195:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1812597010000001, 2024-09-16T22:16:30.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069377570000000001, 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13389 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176573+0000", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176574+0000", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3e-06 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176579+0000", 2024-09-16T22:16:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6110000000000004e-06 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176592+0000", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2721000000000001e-05 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176599+0000", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7149999999999998e-06 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176657+0000", 2024-09-16T22:16:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7246e-05 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.176788+0000", 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131187 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183511+0000", 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067229769999999998 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.441 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13396 3.a 3:57e20437:::rbd_data.11cdc482f8dc.000000000000019a:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1758863689999997, 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065086880000000003, 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13396 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181946+0000", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181947+0000", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4099999999999997e-07 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181952+0000", 2024-09-16T22:16:30.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4170000000000002e-06 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181963+0000", 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1004e-05 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.181968+0000", 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182026+0000", 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8622999999999998e-05 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.182033+0000", 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184627+0000", 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025939729999999999 2024-09-16T22:16:30.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184764+0000", 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013687500000000001 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188455+0000", 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003690891 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13402 3.11 3:89f76da2:::rbd_data.11cdc482f8dc.0000000000000195:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1740617670000004, 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081288179999999995, 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13402 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183771+0000", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183772+0000", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3820000000000001e-06 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183778+0000", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3380000000000004e-06 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183789+0000", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1488000000000001e-05 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183797+0000", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3799999999999996e-06 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183849+0000", 2024-09-16T22:16:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2092000000000002e-05 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.183854+0000", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7019999999999998e-06 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184941+0000", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001086694 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185016+0000", 2024-09-16T22:16:30.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4517000000000005e-05 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185208+0000", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191969 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.187389+0000", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021816829999999998 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191774+0000", 2024-09-16T22:16:30.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043850979999999996 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191843+0000", 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8522999999999994e-05 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191900+0000", 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6952e-05 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13403 3.14 3:2f33ca34:::rbd_data.11cdc482f8dc.0000000000000198:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1738156159999997, 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081506990000000008, 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13403 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184017+0000", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184018+0000", 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0869999999999999e-06 2024-09-16T22:16:30.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184024+0000", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184035+0000", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090000000000001e-05 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184042+0000", 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1280000000000002e-06 2024-09-16T22:16:30.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184245+0000", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000202618 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.184250+0000", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1390000000000003e-06 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185333+0000", 2024-09-16T22:16:30.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010835700000000001 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185402+0000", 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8153999999999998e-05 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.185587+0000", 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000185842 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.191939+0000", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063511710000000001 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192072+0000", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133573 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192130+0000", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8004000000000002e-05 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.192168+0000", 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7574000000000003e-05 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13405 3.a 3:57e20437:::rbd_data.11cdc482f8dc.000000000000019a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1690674779999997, 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091332259999999995, 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13405 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188765+0000", 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188767+0000", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4720000000000001e-06 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188772+0000", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7359999999999998e-06 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188787+0000", 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4790000000000001e-05 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188795+0000", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3440000000000003e-06 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188842+0000", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7547999999999999e-05 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.188944+0000", 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101483 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.189159+0000", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000215439 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194573+0000", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054140380000000004 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197660+0000", 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003086891 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197855+0000", 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019547400000000001 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.197898+0000", 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3010999999999999e-05 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13582 3.a 3:55e8ea82:::rbd_data.11cdc482f8dc.00000000000001f5:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:30.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.0932842459999996, 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069540649999999997, 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13582 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264548+0000", 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264549+0000", 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:30.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264553+0000", 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5530000000000001e-06 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264560+0000", 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4869999999999996e-06 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264568+0000", 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7060000000000004e-06 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264623+0000", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4938000000000002e-05 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264699+0000", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5827000000000004e-05 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.264893+0000", 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019441 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.268738+0000", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038447139999999999 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271425+0000", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026871059999999999 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271469+0000", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3989999999999997e-05 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:30.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.271503+0000", 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3816000000000003e-05 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:30.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:30.472 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:30.473+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:30.572 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:30.574+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:30.575 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:30.576 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:30.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:30.673 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:30.675+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:30.727 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:30.728 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:30.728 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:30.728 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:30.728 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:30.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:30.777 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:30.779+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:30.876 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:30.877+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13034 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16072704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077623919999999, 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020645562999999999, 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13034 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073757+0000", 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6100000000000002e-07 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073761+0000", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.613e-06 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073765+0000", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073767+0000", 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3929999999999998e-06 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094286+0000", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020518814 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094332+0000", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6644999999999998e-05 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094402+0000", 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9738000000000002e-05 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13035 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16109568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077425270000003, 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020762438000000001, 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13035 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.885 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073776+0000", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073777+0000", 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5499999999999999e-07 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073778+0000", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2449999999999999e-06 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073782+0000", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7189999999999999e-06 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073784+0000", 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3989999999999998e-06 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094434+0000", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020649743000000002 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094497+0000", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2924999999999998e-05 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094539+0000", 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2151999999999999e-05 2024-09-16T22:16:30.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13036 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077314399999995, 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020877289, 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13036 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073788+0000", 2024-09-16T22:16:30.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073789+0000", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7300000000000005e-07 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073792+0000", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9009999999999998e-06 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073794+0000", 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2570000000000002e-06 2024-09-16T22:16:30.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094581+0000", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020786782 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094622+0000", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1826999999999998e-05 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094665+0000", 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2348999999999997e-05 2024-09-16T22:16:30.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13037 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077223279999997, 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020988652, 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13037 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073797+0000", 2024-09-16T22:16:30.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0699999999999999e-07 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073798+0000", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1399999999999996e-07 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073801+0000", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8459999999999999e-06 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073803+0000", 2024-09-16T22:16:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.373e-06 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094701+0000", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020898110000000001 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094745+0000", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4134999999999997e-05 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094785+0000", 2024-09-16T22:16:30.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0166999999999997e-05 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13038 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077133510000003, 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021115722999999999, 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13038 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:30.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073806+0000", 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1e-07 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073807+0000", 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0699999999999996e-07 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073810+0000", 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4000000000000001e-06 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073812+0000", 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094818+0000", 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021005830999999999 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094881+0000", 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3137000000000002e-05 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094921+0000", 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0015000000000002e-05 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13039 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16257024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8077038490000001, 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021219551999999999, 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13039 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073815+0000", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073816+0000", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073819+0000", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9059999999999998e-06 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073822+0000", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.977e-06 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094953+0000", 2024-09-16T22:16:30.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021130830999999999 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094992+0000", 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9314999999999999e-05 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095035+0000", 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2514e-05 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13040 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16293888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076942210000002, 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021350451999999999, 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13040 2024-09-16T22:16:30.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073825+0000", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073826+0000", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0800000000000004e-07 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073833+0000", 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.866e-06 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073835+0000", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4030000000000001e-06 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095068+0000", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021233431000000001 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095104+0000", 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5255e-05 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095175+0000", 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1475000000000003e-05 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13041 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16330752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076805580000004, 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021462193000000001, 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13041 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073838+0000", 2024-09-16T22:16:30.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073839+0000", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4400000000000001e-07 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073843+0000", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5449999999999997e-06 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073847+0000", 2024-09-16T22:16:30.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073849+0000", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095216+0000", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021367077000000002 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095258+0000", 2024-09-16T22:16:30.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2027999999999998e-05 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095301+0000", 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2323000000000001e-05 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13042 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076660210000002, 2024-09-16T22:16:30.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021557522999999999, 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13042 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073853+0000", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.36e-07 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073854+0000", 2024-09-16T22:16:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-07 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073857+0000", 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9620000000000002e-06 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073863+0000", 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8619999999999999e-06 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095328+0000", 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021464589999999999 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095367+0000", 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9867999999999998e-05 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095410+0000", 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3102999999999997e-05 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13043 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16404480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076395649999997, 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021632377000000001, 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13043 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073879+0000", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073880+0000", 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073881+0000", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3179999999999999e-06 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073885+0000", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6729999999999998e-06 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073887+0000", 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4590000000000001e-06 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095436+0000", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021548233999999999 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095473+0000", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7478000000000003e-05 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095512+0000", 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804000000000003e-05 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13044 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076285050000003, 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021719948999999999, 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13044 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073890+0000", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073891+0000", 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073895+0000", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9829999999999998e-06 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073898+0000", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1729999999999999e-06 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073900+0000", 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.345e-06 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095535+0000", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021634322000000001 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095571+0000", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095610+0000", 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9387000000000001e-05 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13045 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16478208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076157219999995, 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021805422000000001, 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13045 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073903+0000", 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073904+0000", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9999999999999996e-07 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073907+0000", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9019999999999999e-06 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073910+0000", 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8449999999999999e-06 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095633+0000", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021722769999999999 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095672+0000", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8688e-05 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095709+0000", 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7097000000000001e-05 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13046 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8076045040000004, 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021886407, 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13046 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073914+0000", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073915+0000", 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0800000000000001e-07 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073916+0000", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6400000000000001e-07 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073919+0000", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8410000000000002e-06 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073923+0000", 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1259999999999998e-06 2024-09-16T22:16:30.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095729+0000", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021806635000000001 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095765+0000", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5670999999999998e-05 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095801+0000", 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6062000000000001e-05 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.934 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13047 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16551936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075929219999995, 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021980845999999998, 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13047 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073926+0000", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.11e-07 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073927+0000", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2500000000000004e-07 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073930+0000", 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.943e-06 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073935+0000", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0579999999999997e-06 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095823+0000", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021887747999999999 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095867+0000", 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4206000000000003e-05 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095907+0000", 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9854999999999997e-05 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13048 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16588800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075803730000004, 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022059216, 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13048 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:30.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073939+0000", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.17e-07 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073940+0000", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3099999999999996e-07 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073943+0000", 2024-09-16T22:16:30.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9919999999999999e-06 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073945+0000", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095922+0000", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021977272999999999 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095959+0000", 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6473000000000001e-05 2024-09-16T22:16:30.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095998+0000", 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9183000000000001e-05 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13049 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16625664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075713100000002, 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022143023000000001, 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13049 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073948+0000", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0800000000000001e-07 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073949+0000", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1800000000000004e-07 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073952+0000", 2024-09-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.393e-06 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073955+0000", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3470000000000001e-06 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096014+0000", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022059055000000001 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096052+0000", 2024-09-16T22:16:30.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8871000000000001e-05 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096091+0000", 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8231e-05 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13050 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.807546522, 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022205784999999999, 2024-09-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13050 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073972+0000", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073973+0000", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8700000000000001e-07 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073974+0000", 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2109999999999999e-06 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073977+0000", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.067e-06 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073980+0000", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3920000000000001e-06 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096104+0000", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022124516 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096140+0000", 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5537000000000002e-05 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096178+0000", 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8674999999999998e-05 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13051 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16699392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:30.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075359730000002, 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022283160999999999, 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13051 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073983+0000", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073984+0000", 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.16e-07 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073987+0000", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9019999999999999e-06 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073994+0000", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.221e-06 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096193+0000", 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022198401 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096230+0000", 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7231000000000003e-05 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096266+0000", 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6180999999999998e-05 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13052 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16736256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075215169999996, 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022365262, 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13052 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073997+0000", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073998+0000", 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.073999+0000", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3600000000000002e-07 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074004+0000", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8900000000000004e-06 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074007+0000", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096290+0000", 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022282381 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096326+0000", 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5824000000000001e-05 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096363+0000", 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7184999999999997e-05 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13053 3.6 3:6200f3c8:::rbd_data.11cd2cec779f.0000000000000001:6 [read 16773120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8075085079999997, 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022442749000000001, 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13053 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074010+0000", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074011+0000", 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.16e-07 2024-09-16T22:16:30.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074014+0000", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074017+0000", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2310000000000001e-06 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.074020+0000", 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3439999999999999e-06 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096379+0000", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022359875000000001 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096416+0000", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6431999999999999e-05 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.096453+0000", 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7385999999999997e-05 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:30.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:30.964 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:30.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:30.977 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:30.979+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:31.078 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:31.080+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:31.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:31.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:31.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:31.179 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:31.181+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:31.203 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:31.203 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:31.203 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T22:16:31.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T22:16:31.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:16:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:31.223 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:31.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:31.307 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T22:16:31.317 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:31.319+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:31.380 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:31.382+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4201.0:90 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=17'60)", 2024-09-16T22:16:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.253962968000003, 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056948570000000002, 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130102+0000", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130103+0000", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130163+0000", 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0516999999999997e-05 2024-09-16T22:16:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130172+0000", 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3990000000000004e-06 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130177+0000", 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8960000000000002e-06 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130210+0000", 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3492999999999999e-05 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.130220+0000", 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0176999999999999e-05 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135746+0000", 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055257730000000003 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:15:59.135797+0000", 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0982000000000001e-05 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.249176543000001, 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095371610000000006, 2024-09-16T22:16:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 11 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134889+0000", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134890+0000", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.556e-06 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134902+0000", 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1999e-05 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134914+0000", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1233000000000001e-05 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134919+0000", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.48e-06 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134968+0000", 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8915000000000002e-05 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.134972+0000", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7929999999999999e-06 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135632+0000", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065986099999999998 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135741+0000", 2024-09-16T22:16:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109416 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.135889+0000", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014819200000000001 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.137526+0000", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001637129 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144330+0000", 2024-09-16T22:16:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068036010000000003 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144379+0000", 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9141999999999998e-05 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:17.144426+0000", 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6844000000000001e-05 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1894 3.0 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.5888692560000006, 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006210162, 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1894 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795196+0000", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795197+0000", 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795202+0000", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7190000000000001e-06 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795215+0000", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3383e-05 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795223+0000", 2024-09-16T22:16:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8450000000000003e-06 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795287+0000", 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3838999999999996e-05 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795368+0000", 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0927000000000006e-05 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795486+0000", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011838800000000001 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.796188+0000", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070161500000000005 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801350+0000", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051619099999999996 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801379+0000", 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9549000000000001e-05 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801406+0000", 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6996999999999999e-05 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1916 3.0 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.553117276, 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015511264, 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1916 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830948+0000", 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830949+0000", 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830953+0000", 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.476e-06 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830964+0000", 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529e-05 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.830969+0000", 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2519999999999999e-06 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.831008+0000", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8915999999999998e-05 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.831057+0000", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9146e-05 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.833521+0000", 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024641960000000001 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846354+0000", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012832427 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846392+0000", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8724999999999999e-05 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846428+0000", 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5593000000000003e-05 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.846459+0000", 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1269999999999997e-05 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:8552 3.17 3:ee12f636:::rbd_data.11cdc482f8dc.000000000000007e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7080273019999996, 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006238637, 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 8552 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676038+0000", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676039+0000", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676046+0000", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4189999999999999e-06 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676058+0000", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2566e-05 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676066+0000", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3650000000000004e-06 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676116+0000", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9981000000000001e-05 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676178+0000", 2024-09-16T22:16:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1494000000000004e-05 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676390+0000", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021231900000000001 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.676824+0000", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043400399999999999 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682223+0000", 2024-09-16T22:16:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053988350000000003 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682255+0000", 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1813000000000002e-05 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.682277+0000", 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1661000000000001e-05 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9869 3.1e 3:7faa808b:::rbd_data.11cdc482f8dc.00000000000000b2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5800687629999999, 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068475999999999997, 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9869 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.803997+0000", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0500000000000002e-07 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804003+0000", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.169e-06 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804015+0000", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2856e-05 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804023+0000", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5900000000000002e-06 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804069+0000", 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5568999999999997e-05 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804115+0000", 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6455000000000002e-05 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804238+0000", 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122967 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.804710+0000", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000472483 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810720+0000", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060092699999999997 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810805+0000", 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4958999999999998e-05 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810844+0000", 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9376999999999999e-05 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9900 3.1e 3:7fed7f31:::rbd_data.11cdc482f8dc.00000000000000c5:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5623994159999999, 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058095239999999999, 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9900 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821666+0000", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821667+0000", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3200000000000004e-07 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821671+0000", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0840000000000002e-06 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821680+0000", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.799e-06 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821685+0000", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821722+0000", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7854e-05 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.821792+0000", 2024-09-16T22:16:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0006999999999996e-05 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.827475+0000", 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056828929999999996 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:10823 3.1a e31/23 3:5ab4cea0:::rbd_data.11cdc482f8dc.00000000000000ca:head v 31'52, mlcod=31'50)", 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4765641120000002, 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005731603, 2024-09-16T22:16:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907501+0000", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907502+0000", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1099999999999995e-07 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907507+0000", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2279999999999998e-06 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907519+0000", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907526+0000", 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.55e-06 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907580+0000", 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.907602+0000", 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2166000000000001e-05 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.913191+0000", 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055893219999999999 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.913233+0000", 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1476000000000001e-05 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13229 3.1a e31/23 3:582e74af:::rbd_data.11cdc482f8dc.0000000000000120:head v 31'63, mlcod=31'61)", 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2728795660000003, 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058645540000000001, 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111186+0000", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111187+0000", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111192+0000", 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8939999999999998e-06 2024-09-16T22:16:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111203+0000", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0560000000000001e-05 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111209+0000", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4479999999999999e-06 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111270+0000", 2024-09-16T22:16:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0690999999999999e-05 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111290+0000", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0163999999999999e-05 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.117009+0000", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057184690000000003 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.117050+0000", 2024-09-16T22:16:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1517999999999998e-05 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13239 3.18 3:1836d7a2:::rbd_data.11cdc482f8dc.000000000000014c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2696300169999999, 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057377039999999997, 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13239 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114435+0000", 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114436+0000", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114443+0000", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7320000000000001e-06 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114453+0000", 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9110000000000005e-06 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114459+0000", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.798e-06 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114508+0000", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8621000000000001e-05 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114628+0000", 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120402 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120173+0000", 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005544546 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13257 3.18 3:1836d7a2:::rbd_data.11cdc482f8dc.000000000000014c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2632436800000004, 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075231200000000003, 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13257 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120822+0000", 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120826+0000", 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9060000000000004e-06 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120842+0000", 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5800000000000001e-05 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120847+0000", 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8709999999999996e-06 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120887+0000", 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0488000000000003e-05 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120941+0000", 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3610999999999999e-05 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.121042+0000", 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101389 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.125714+0000", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046714859999999999 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128169+0000", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024548880000000001 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128307+0000", 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013875700000000001 2024-09-16T22:16:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128345+0000", 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7304000000000001e-05 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13258 3.1e 3:7eafc415:::rbd_data.11cdc482f8dc.0000000000000154:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2631823940000002, 2024-09-16T22:16:31.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074323890000000002, 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13258 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:31.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120883+0000", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120886+0000", 2024-09-16T22:16:31.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1550000000000001e-06 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120890+0000", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7620000000000003e-06 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120894+0000", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3720000000000001e-06 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120934+0000", 2024-09-16T22:16:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0088e-05 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120999+0000", 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104000000000006e-05 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.128315+0000", 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073164140000000003 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13378 3.0 3:06bf120b:::rbd_data.11cdc482f8dc.000000000000018b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2133576760000002, 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057724650000000001, 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13378 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170708+0000", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170709+0000", 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0189999999999999e-06 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170714+0000", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170727+0000", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.274e-05 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170734+0000", 2024-09-16T22:16:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6159999999999996e-06 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170783+0000", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8437999999999999e-05 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170843+0000", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0201000000000002e-05 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170958+0000", 2024-09-16T22:16:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114961 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.173462+0000", 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002503917 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176392+0000", 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029305030000000001 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176443+0000", 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0800000000000002e-05 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176480+0000", 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6950999999999999e-05 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13387 3.1e 3:7b9a15c0:::rbd_data.11cdc482f8dc.0000000000000190:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2089024110000004, 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080670580000000002, 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13387 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175163+0000", 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175164+0000", 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6000000000000002e-07 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175171+0000", 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175182+0000", 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0926e-05 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175187+0000", 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-09-16T22:16:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175224+0000", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6868e-05 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175269+0000", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.515e-05 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.175374+0000", 2024-09-16T22:16:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104876 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179182+0000", 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003808287 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183166+0000", 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039836710000000003 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183203+0000", 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200999999999998e-05 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183230+0000", 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7151000000000001e-05 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13434 3.17 3:ea1a1df2:::rbd_data.11cdc482f8dc.00000000000001a8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.178817757, 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060193850000000004, 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13434 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205248+0000", 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5199999999999995e-07 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205253+0000", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7169999999999999e-06 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205265+0000", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2216e-05 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205273+0000", 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6580000000000002e-06 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205323+0000", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0488999999999997e-05 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205395+0000", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1654000000000002e-05 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.205579+0000", 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183926 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206631+0000", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010517129999999999 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211212+0000", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045817269999999998 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211244+0000", 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1452999999999999e-05 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211267+0000", 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.298e-05 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13454 3.17 3:ed04fddc:::rbd_data.11cdc482f8dc.00000000000001b3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1721607790000004, 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073096990000000002, 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13454 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211904+0000", 2024-09-16T22:16:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211905+0000", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211909+0000", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0210000000000001e-06 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211917+0000", 2024-09-16T22:16:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211923+0000", 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9800000000000003e-06 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211973+0000", 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9599999999999999e-05 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212040+0000", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7057999999999994e-05 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212241+0000", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020084200000000001 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.213002+0000", 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076082000000000005 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219131+0000", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061289140000000001 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219179+0000", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8312000000000001e-05 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219214+0000", 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5275999999999999e-05 2024-09-16T22:16:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.10 epoch 32/23 rep_tid 221 entries 32'35 (0'0) error 3:08d88565:::rbd_data.11cdc482f8dc.00000000000001b8:head by client.4557.0:13455 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'33)", 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1716415859999998, 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067463319999999998, 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212424+0000", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4699999999999998e-07 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212429+0000", 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212447+0000", 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7986e-05 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212461+0000", 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4149e-05 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212542+0000", 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0692000000000006e-05 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219170+0000", 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066281530000000003 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13471 3.1e 3:788c3ff6:::rbd_data.11cdc482f8dc.00000000000001c0:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1642905160000003, 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056951670000000001, 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13471 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219775+0000", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219776+0000", 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1200000000000001e-07 2024-09-16T22:16:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219780+0000", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2309999999999998e-06 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219790+0000", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0122000000000001e-05 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219797+0000", 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9e-06 2024-09-16T22:16:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219862+0000", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5093999999999997e-05 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219989+0000", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012717100000000001 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225470+0000", 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054807370000000003 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:14433 3.1e 3:7bfca97c:::rbd_data.11cd4c15da38.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:27.930270+0000", 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.4537956830000001, 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020143990000000001, 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 14433 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930270+0000", 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930270+0000", 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930271+0000", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.406e-06 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930277+0000", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8139999999999997e-06 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930289+0000", 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1931e-05 2024-09-16T22:16:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930294+0000", 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.31e-06 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930340+0000", 2024-09-16T22:16:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5932e-05 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.930422+0000", 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1736000000000005e-05 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.933608+0000", 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031863680000000002 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950281+0000", 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016672590000000001 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950339+0000", 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.791e-05 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950378+0000", 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9889999999999999e-05 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950414+0000", 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5102999999999999e-05 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4557.0:14433 3.1e e33/23 ondisk, result = 0)", 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:27.943670+0000", 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.4403953600000001, 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067502710000000004, 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943670+0000", 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943670+0000", 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943671+0000", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9800000000000002e-07 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943676+0000", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0610000000000003e-06 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943693+0000", 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7377999999999999e-05 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.943704+0000", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0674e-05 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950326+0000", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066223710000000002 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950334+0000", 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-09-16T22:16:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.950420+0000", 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5828000000000006e-05 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.486 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:31.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1094 punch from 0x9a61d2 to 0x9a9d98, (0x3bc6 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1095 punch from 0xfea16b to 0xff3d9f, (0x9c34 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1097 read 0x4f5ecf thru 0x4f6098 (0x1ca bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1098 punch from 0xa93326 to 0xa9b2d8, (0x7fb2 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1099 writesame 0x9fb51bc thru 0x9fba459 data_size 0x6(0x529e bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1102 writesame 0x33dd6fd thru 0x33dfefc data_size 0x1(0x2800 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1103 read 0x3432b0a thru 0x34429ec (0xfee3 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1104 write 0x1afb4fe thru 0x1afe295 (0x2d98 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1105 read 0x4dd0635 thru 0x4ddddc8 (0xd794 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1106 trunc from 0x9fba45a to 0xddb72d8 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1107 punch from 0x757844c to 0x758667f, (0xe233 bytes) 2024-09-16T22:16:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:1109 read 0x9aaf337 thru 0x9aba9d8 (0xb6a2 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1111 punch from 0x728fed6 to 0x72965f8, (0x6722 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1112 punch from 0x898c2d5 to 0x8997cc9, (0xb9f4 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1115 trunc from 0xddb72d8 to 0x66d4fd0 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1117 read 0x534325 thru 0x53bb74 (0x7850 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1118 read 0x3bc0178 thru 0x3bc1067 (0xef0 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1119 read 0xa34eec thru 0xa42ad9 (0xdbee bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1122 punch from 0x757690 to 0x75798b, (0x2fb bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1123 write 0x43df175 thru 0x43e18dc (0x2768 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1124 read 0x3badfd8 thru 0x3bb676f (0x8798 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1125 punch from 0x39c819e to 0x39d0fde, (0x8e40 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1126 read 0x316f790 thru 0x3175e92 (0x6703 bytes) 2024-09-16T22:16:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:1127 trunc from 0x66d4fd0 to 0x6d28d59 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1128 read 0x216337c thru 0x21669ee (0x3673 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1129 read 0x6b96b13 thru 0x6b99ee2 (0x33d0 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1130 read 0x3c00061 thru 0x3c02886 (0x2826 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1131 trunc from 0x6d28d59 to 0xa579244 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1133 read 0x2572c5d thru 0x2582bae (0xff52 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1134 writesame 0xeaf4232 thru 0xeaf5d61 data_size 0xa(0x1b30 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1135 write 0x41e7247 thru 0x41f1f7b (0xad35 bytes) 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1136 writesame 0x5beaf4c thru 0x5bedee6 data_size 0x1(0x2f9b bytes) 2024-09-16T22:16:31.493 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:31.489+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:1141 writesame 0x3ec830e thru 0x3ecacbb data_size 0x2(0x29ae bytes) 2024-09-16T22:16:31.495 INFO:teuthology.orchestra.run.smithi031.stdout:1142 punch from 0xb8e2127 to 0xb8eec51, (0xcb2a bytes) 2024-09-16T22:16:31.495 INFO:teuthology.orchestra.run.smithi031.stdout:1143 read 0xac6206 thru 0xacefe1 (0x8ddc bytes) 2024-09-16T22:16:31.495 INFO:teuthology.orchestra.run.smithi031.stdout:1145 read 0xaf45eea thru 0xaf5563d (0xf754 bytes) 2024-09-16T22:16:31.496 INFO:teuthology.orchestra.run.smithi031.stdout:1146 read 0x241d6ce thru 0x242a23f (0xcb72 bytes) 2024-09-16T22:16:31.501 INFO:teuthology.orchestra.run.smithi031.stdout:1147 write 0xecd6040 thru 0xece2a35 (0xc9f6 bytes) 2024-09-16T22:16:31.504 INFO:teuthology.orchestra.run.smithi031.stdout:1149 write 0x72da7b0 thru 0x72e04ba (0x5d0b bytes) 2024-09-16T22:16:31.507 INFO:teuthology.orchestra.run.smithi031.stdout:1150 write 0x7fad621 thru 0x7fb6584 (0x8f64 bytes) 2024-09-16T22:16:31.508 INFO:teuthology.orchestra.run.smithi031.stdout:1151 writesame 0xac08838 thru 0xac128c1 data_size 0x1(0xa08a bytes) 2024-09-16T22:16:31.530 INFO:teuthology.orchestra.run.smithi031.stdout:1154 write 0xb3285c5 thru 0xb32fb14 (0x7550 bytes) 2024-09-16T22:16:31.533 INFO:teuthology.orchestra.run.smithi031.stdout:1155 write 0xe2adc21 thru 0xe2b1f46 (0x4326 bytes) 2024-09-16T22:16:31.535 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.535 INFO:teuthology.orchestra.run.smithi031.stdout:1157 write 0x5dd61fb thru 0x5dde30a (0x8110 bytes) 2024-09-16T22:16:31.537 INFO:teuthology.orchestra.run.smithi031.stdout:1160 read 0x9699a64 thru 0x969a129 (0x6c6 bytes) 2024-09-16T22:16:31.537 INFO:teuthology.orchestra.run.smithi031.stdout:1161 read 0xd5e5937 thru 0xd5f14e9 (0xbbb3 bytes) 2024-09-16T22:16:31.537 INFO:teuthology.orchestra.run.smithi031.stdout:1162 writesame 0x683e8bf thru 0x68434fd data_size 0x1(0x4c3f bytes) 2024-09-16T22:16:31.550 INFO:teuthology.orchestra.run.smithi031.stdout:1163 punch from 0xc83e76a to 0xc841d99, (0x362f bytes) 2024-09-16T22:16:31.550 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:31.550 INFO:teuthology.orchestra.run.smithi031.stdout:1165 read 0xe525024 thru 0xe53260f (0xd5ec bytes) 2024-09-16T22:16:31.551 INFO:teuthology.orchestra.run.smithi031.stdout:1166 writesame 0x4f198c1 thru 0x4f1f051 data_size 0x1(0x5791 bytes) 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 35, 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-16T22:15:44.319460+0000", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-16T22:16:31.193528+0000", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-16T22:15:56.374983+0000", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-16T22:15:46.438697+0000", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-16T22:16:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-16T22:16:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:15:56.583719+0000", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-16T22:16:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-16T22:16:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-16T22:16:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:02.234097+0000", 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "22", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 22, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:13.789572+0000", 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "35", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 9, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 35, 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "44de8de0-e16a-4325-94b9-1cc61f4829db", 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2585124124", 2024-09-16T22:16:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2585124124", 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2585124124", 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2585124124", 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0e05fbd1-9707-40b8-91a3-198c6f5b7865", 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-16T22:16:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3271285223", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3271285223", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3271285223", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3271285223", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "b080bc29-0a92-4200-ae3f-0a17a1722145", 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.9312591552734375, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2874892035", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2874892035", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2874892035", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2874892035", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0bc64c75-486c-40ac-abaf-381e151623bd", 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3768411773", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3768411773", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3768411773", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3768411773", 2024-09-16T22:16:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fae627d1-6bc2-4e5b-85e1-334e5e05f830", 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6824", 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6825", 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6826", 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6827", 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6830", 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6831", 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6828", 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6829", 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6825/3421901083", 2024-09-16T22:16:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6827/3421901083", 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3421901083", 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6829/3421901083", 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "989a79cd-ef84-4bf3-a64f-9d101d0b29d7", 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6816", 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6817", 2024-09-16T22:16:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6818", 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6819", 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6822", 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6823", 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6820", 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6821", 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6817/335833224", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6819/335833224", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6823/335833224", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6821/335833224", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "92f145b7-e868-43fd-9abe-68c43a407ae2", 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6808", 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6809", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6810", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6811", 2024-09-16T22:16:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6814", 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6815", 2024-09-16T22:16:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6812", 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6813", 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6809/1592419018", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6811/1592419018", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1592419018", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1592419018", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "373f8002-53a4-4ca7-bbbf-70e8d56b3571", 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6800", 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6801", 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6802", 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6803", 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6806", 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6807", 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6804", 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6805", 2024-09-16T22:16:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6801/3448871901", 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6803/3448871901", 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3448871901", 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3448871901", 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.033837+0000", 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.799099+0000", 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.769970+0000", 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.965029+0000", 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.706036+0000", 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.158188+0000", 2024-09-16T22:16:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.722931+0000", 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.485009+0000", 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-16T22:16:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-16T22:16:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:31.635 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:31.636 INFO:teuthology.orchestra.run.smithi031.stdout:1168 writesame 0x92fbe4c thru 0x9300b82 data_size 0xb(0x4d37 bytes) 2024-09-16T22:16:31.636 INFO:teuthology.orchestra.run.smithi031.stdout:1169 read 0xec85731 thru 0xec8814f (0x2a1f bytes) 2024-09-16T22:16:31.636 INFO:teuthology.orchestra.run.smithi031.stdout:1170 write 0x2cecee8 thru 0x2cf9858 (0xc971 bytes) 2024-09-16T22:16:31.636 INFO:teuthology.orchestra.run.smithi031.stdout:1171 writesame 0x732ba48 thru 0x7331458 data_size 0x1(0x5a11 bytes) 2024-09-16T22:16:31.638 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '57fe7da5-e356-48a4-ba33-5558b1026a32', 'created': '2024-09-16T22:15:44.319460+0000', 'modified': '2024-09-16T22:16:31.193528+0000', 'last_up_change': '2024-09-16T22:15:56.374983+0000', 'last_in_change': '2024-09-16T22:15:46.438697+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T22:15:56.583719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T22:16:02.234097+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T22:16:13.789572+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 35, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '44de8de0-e16a-4325-94b9-1cc61f4829db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2585124124}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2585124124}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2585124124}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2585124124}]}, 'public_addr': '172.21.15.31:6817/2585124124', 'cluster_addr': '172.21.15.31:6819/2585124124', 'heartbeat_back_addr': '172.21.15.31:6823/2585124124', 'heartbeat_front_addr': '172.21.15.31:6821/2585124124', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3271285223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3271285223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3271285223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3271285223}]}, 'public_addr': '172.21.15.31:6801/3271285223', 'cluster_addr': '172.21.15.31:6803/3271285223', 'heartbeat_back_addr': '172.21.15.31:6807/3271285223', 'heartbeat_front_addr': '172.21.15.31:6805/3271285223', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b080bc29-0a92-4200-ae3f-0a17a1722145', 'up': 1, 'in': 1, 'weight': 0.9312591552734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2874892035}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2874892035}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2874892035}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2874892035}]}, 'public_addr': '172.21.15.31:6809/2874892035', 'cluster_addr': '172.21.15.31:6811/2874892035', 'heartbeat_back_addr': '172.21.15.31:6815/2874892035', 'heartbeat_front_addr': '172.21.15.31:6813/2874892035', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0bc64c75-486c-40ac-abaf-381e151623bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3768411773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3768411773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3768411773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3768411773}]}, 'public_addr': '172.21.15.31:6825/3768411773', 'cluster_addr': '172.21.15.31:6827/3768411773', 'heartbeat_back_addr': '172.21.15.31:6831/3768411773', 'heartbeat_front_addr': '172.21.15.31:6829/3768411773', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 3421901083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3421901083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3421901083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3421901083}]}, 'public_addr': '172.21.15.92:6825/3421901083', 'cluster_addr': '172.21.15.92:6827/3421901083', 'heartbeat_back_addr': '172.21.15.92:6831/3421901083', 'heartbeat_front_addr': '172.21.15.92:6829/3421901083', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 335833224}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 335833224}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 335833224}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 335833224}]}, 'public_addr': '172.21.15.92:6817/335833224', 'cluster_addr': '172.21.15.92:6819/335833224', 'heartbeat_back_addr': '172.21.15.92:6823/335833224', 'heartbeat_front_addr': '172.21.15.92:6821/335833224', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '92f145b7-e868-43fd-9abe-68c43a407ae2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1592419018}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1592419018}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1592419018}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1592419018}]}, 'public_addr': '172.21.15.92:6809/1592419018', 'cluster_addr': '172.21.15.92:6811/1592419018', 'heartbeat_back_addr': '172.21.15.92:6815/1592419018', 'heartbeat_front_addr': '172.21.15.92:6813/1592419018', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '373f8002-53a4-4ca7-bbbf-70e8d56b3571', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3448871901}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3448871901}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3448871901}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3448871901}]}, 'public_addr': '172.21.15.92:6801/3448871901', 'cluster_addr': '172.21.15.92:6803/3448871901', 'heartbeat_back_addr': '172.21.15.92:6807/3448871901', 'heartbeat_front_addr': '172.21.15.92:6805/3448871901', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.033837+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.799099+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.769970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.965029+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.706036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.158188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.722931+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.485009+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T22:16:31.638 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T22:16:31.638 INFO:teuthology.orchestra.run.smithi031.stdout:1172 writesame 0xc560bc5 thru 0xc56d5bf data_size 0x1(0xc9fb bytes) 2024-09-16T22:16:31.638 INFO:teuthology.orchestra.run.smithi031.stdout:1173 write 0x466b6d3 thru 0x4674b6e (0x949c bytes) 2024-09-16T22:16:31.638 INFO:teuthology.orchestra.run.smithi031.stdout:1174 read 0x2e5a4f5 thru 0x2e66029 (0xbb35 bytes) 2024-09-16T22:16:31.638 INFO:teuthology.orchestra.run.smithi031.stdout:1175 trunc from 0xece2a36 to 0x310772c 2024-09-16T22:16:31.639 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:31.638+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:31.639 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:31.639 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:31.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:31.662 INFO:teuthology.orchestra.run.smithi031.stdout:1177 write 0x732469e thru 0x7328348 (0x3cab bytes) 2024-09-16T22:16:31.664 INFO:teuthology.orchestra.run.smithi031.stdout:1178 read 0x3c7fb04 thru 0x3c8bc65 (0xc162 bytes) 2024-09-16T22:16:31.673 INFO:teuthology.orchestra.run.smithi031.stdout:1179 writesame 0xa3d92d0 thru 0xa3e7564 data_size 0xf(0xe295 bytes) 2024-09-16T22:16:31.681 INFO:teuthology.orchestra.run.smithi031.stdout:1180 trunc from 0xa3e7565 to 0x322d277 2024-09-16T22:16:31.711 INFO:teuthology.orchestra.run.smithi031.stdout:1181 write 0xe91f557 thru 0xe92b5d3 (0xc07d bytes) 2024-09-16T22:16:31.715 INFO:teuthology.orchestra.run.smithi031.stdout:1182 trunc from 0xe92b5d4 to 0xd68b6bb 2024-09-16T22:16:31.724 INFO:teuthology.orchestra.run.smithi031.stdout:1183 read 0x361dd0b thru 0x3628e28 (0xb11e bytes) 2024-09-16T22:16:31.724 INFO:teuthology.orchestra.run.smithi031.stdout:1184 clone 8 order 22 su 16384 sc 5 2024-09-16T22:16:31.736 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:31.738+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.791 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:31.791 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:31.791 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:31.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:31.841 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:31.843+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:31.938 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:31.940+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1373 3.b e28/23 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head v 28'7, mlcod=27'5)", 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8160175019999993, 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057964999999999996, 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9700000000000003e-07 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129215+0000", 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8830000000000002e-06 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129222+0000", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8469999999999998e-06 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129227+0000", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4870000000000004e-06 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129279+0000", 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2219000000000003e-05 2024-09-16T22:16:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129299+0000", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9877999999999998e-05 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.134962+0000", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056633079999999997 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135009+0000", 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6581e-05 2024-09-16T22:16:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3524492539999997, 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:16:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:16:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:16:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1818 3.1 3:8772e73b:::rbd_data.11cd70c2eac3.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2246922560000009, 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062717169999999996, 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1818 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720539+0000", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720543+0000", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.464e-06 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720552+0000", 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3090000000000001e-06 2024-09-16T22:16:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720560+0000", 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6909999999999997e-06 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720604+0000", 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3838999999999998e-05 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720683+0000", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8916000000000001e-05 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.721544+0000", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086141500000000003 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726702+0000", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005157671 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726720+0000", 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717999999999999e-05 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726768+0000", 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8035999999999999e-05 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726809+0000", 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1539000000000003e-05 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1933 3.12 3:4b562b7a:::rbd_data.11cd70c2eac3.000000000000000d:head [truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.0801392770000007, 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056569769999999997, 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1933 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865092+0000", 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6309999999999999e-06 2024-09-16T22:16:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865097+0000", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865116+0000", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8766e-05 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865124+0000", 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0099999999999995e-06 2024-09-16T22:16:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865179+0000", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5169e-05 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865257+0000", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8221999999999997e-05 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865377+0000", 2024-09-16T22:16:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119473 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.867211+0000", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00183428 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870680+0000", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003468563 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870710+0000", 2024-09-16T22:16:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0337000000000001e-05 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870747+0000", 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7228000000000003e-05 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9870 3.1 3:8781002f:::rbd_data.11cdc482f8dc.00000000000000b6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.1401110680000004, 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057426480000000004, 2024-09-16T22:16:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9870 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805127+0000", 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6860000000000006e-06 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805137+0000", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7000000000000003e-06 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805144+0000", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805215+0000", 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1155999999999994e-05 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805261+0000", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6279000000000002e-05 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805365+0000", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103929 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805721+0000", 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035559099999999999 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810764+0000", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050430689999999999 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810817+0000", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3029999999999999e-05 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810861+0000", 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4372999999999997e-05 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:12107 3.4 3:202b7b59:::rbd_data.11cdc482f8dc.0000000000000116:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9454387090000003, 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054483370000000001, 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12107 2024-09-16T22:16:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999792+0000", 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9300000000000006e-07 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999796+0000", 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4279999999999998e-06 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999807+0000", 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0824e-05 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999813+0000", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9789999999999998e-06 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999863+0000", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0127000000000003e-05 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999976+0000", 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112828 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.005239+0000", 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052631580000000004 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13077 3.4 3:26c5fb8d:::rbd_data.11cdc482f8dc.0000000000000133:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8504405359999998, 2024-09-16T22:16:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053770570000000002, 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13077 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094790+0000", 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7499999999999999e-07 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094794+0000", 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.907e-06 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094806+0000", 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.221e-05 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094813+0000", 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1969999999999999e-06 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094864+0000", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1103000000000002e-05 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094911+0000", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7077999999999999e-05 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095014+0000", 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102525 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100050+0000", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050360680000000003 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100075+0000", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4459000000000001e-05 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100134+0000", 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9725000000000002e-05 2024-09-16T22:16:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100166+0000", 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1909999999999998e-05 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13225 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8367433670000004, 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056928450000000002, 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13225 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108487+0000", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6899999999999996e-07 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108498+0000", 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0733999999999999e-05 2024-09-16T22:16:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108509+0000", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553999999999999e-05 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108515+0000", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9480000000000002e-06 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108574+0000", 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8906000000000001e-05 2024-09-16T22:16:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108713+0000", 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013846 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114179+0000", 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054662740000000001 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13240 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8306922590000001, 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057273300000000001, 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13240 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114538+0000", 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114542+0000", 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2710000000000003e-06 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114552+0000", 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6029999999999994e-06 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114564+0000", 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1555e-05 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114611+0000", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7686999999999999e-05 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114691+0000", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9938000000000006e-05 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114862+0000", 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017119999999999999 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.119890+0000", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050279849999999996 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120170+0000", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027961400000000002 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120224+0000", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3922000000000003e-05 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120265+0000", 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0760000000000003e-05 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13339 3.12 3:4fd424ff:::rbd_data.11cdc482f8dc.0000000000000179:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7945754770000004, 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065394440000000002, 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13339 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150656+0000", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150665+0000", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8529999999999998e-06 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150685+0000", 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0222999999999999e-05 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150697+0000", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150774+0000", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6954999999999998e-05 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150917+0000", 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014322900000000001 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151076+0000", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015856799999999999 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151448+0000", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037228599999999999 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157138+0000", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056900379999999997 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157169+0000", 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0910000000000001e-05 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157194+0000", 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4542999999999999e-05 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13353 3.1 3:80996581:::rbd_data.11cdc482f8dc.0000000000000185:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7870549090000001, 2024-09-16T22:16:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054098480000000001, 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13353 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158176+0000", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158180+0000", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158189+0000", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9190000000000005e-06 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158196+0000", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8340000000000001e-06 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158243+0000", 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7157999999999998e-05 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158319+0000", 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5574000000000005e-05 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.163585+0000", 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052660169999999996 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13391 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7685775049999997, 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066380959999999996, 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13391 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176653+0000", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.305e-06 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176658+0000", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176670+0000", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1982e-05 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176675+0000", 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1220000000000003e-06 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176719+0000", 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4036000000000003e-05 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176798+0000", 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8981999999999997e-05 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183290+0000", 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064919390000000004 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.761541706, 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:16:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:16:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.5 epoch 32/23 rep_tid 154 entries 32'60 (0'0) error 3:a07194e2:::rbd_data.11cdc482f8dc.00000000000001b1:head by client.4557.0:13444 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 32'58)", 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.208807+0000", 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7364225329999998, 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005315579, 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208807+0000", 2024-09-16T22:16:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208807+0000", 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208808+0000", 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208816+0000", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.875e-06 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208836+0000", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0193000000000001e-05 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208845+0000", 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4669999999999993e-06 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.208911+0000", 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5951000000000006e-05 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.214123+0000", 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052114170000000003 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13466 3.12 3:49b34a3f:::rbd_data.11cdc482f8dc.00000000000001bc:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.725813821, 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055848399999999998, 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13466 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219419+0000", 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6860000000000002e-06 2024-09-16T22:16:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219426+0000", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0480000000000001e-06 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219431+0000", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219939+0000", 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050803999999999997 2024-09-16T22:16:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.220051+0000", 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011102700000000001 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225001+0000", 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049501299999999996 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7110270170000001, 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:16:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.6977009269999996, 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:16:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:16:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:16:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13595 3.5 e32/23 3:a7f0f76d:::rbd_data.11cdc482f8dc.00000000000001fd:head v 32'67, mlcod=32'65)", 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.6747345149999999, 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053849240000000001, 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270496+0000", 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270500+0000", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0350000000000003e-06 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270508+0000", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4970000000000001e-06 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270513+0000", 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7720000000000002e-06 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270549+0000", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5667000000000003e-05 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270567+0000", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8507000000000001e-05 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275827+0000", 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005260069 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275880+0000", 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2831000000000003e-05 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.35620241499999999, 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:16:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4557.0:16275 3.4 e35/23 ondisk, result = 0)", 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.33961990199999997, 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084889960000000004, 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605611+0000", 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3009999999999999e-06 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605616+0000", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605631+0000", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4827000000000001e-05 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605640+0000", 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3840000000000003e-06 2024-09-16T22:16:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614068+0000", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084278679999999998 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614081+0000", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2576e-05 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614099+0000", 2024-09-16T22:16:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8108e-05 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.043 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:32.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:32.045 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:32.046+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:32.145 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:32.147+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:32.187 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:32.187 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:32.188 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:32.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:16:32.245 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:32.247+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:32.273 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone7 from 0xd68b6bb (overlap 0x10496c9) to 0x4e2d3c 2024-09-16T22:16:32.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:32.310 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:32.310 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:32.310 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:32.310 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:32.310 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:32.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:32.326 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T22:16:32.347 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:32.348+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:32.349 INFO:teuthology.orchestra.run.smithi031.stdout:1185 punch from 0xc77eff8 to 0xc78c3f7, (0xd3ff bytes) 2024-09-16T22:16:32.352 INFO:teuthology.orchestra.run.smithi031.stdout:1186 punch from 0x2a2cc46 to 0x2a3a19b, (0xd555 bytes) 2024-09-16T22:16:32.352 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:32.356 INFO:teuthology.orchestra.run.smithi031.stdout:1189 trunc from 0xd68b6bb to 0x9e3bd24 2024-09-16T22:16:32.368 INFO:teuthology.orchestra.run.smithi031.stdout:1190 read 0x287c01f thru 0x2886d67 (0xad49 bytes) 2024-09-16T22:16:32.369 INFO:teuthology.orchestra.run.smithi031.stdout:1194 read 0x168e07f thru 0x1699eb8 (0xbe3a bytes) 2024-09-16T22:16:32.370 INFO:teuthology.orchestra.run.smithi031.stdout:1195 writesame 0x36aed3c thru 0x36b8110 data_size 0x57(0x93d5 bytes) 2024-09-16T22:16:32.448 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:32.449+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1373 3.b 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.33912503, 2024-09-16T22:16:32.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064652499999999996, 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1373 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0299999999999998e-07 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128763+0000", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-06 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128773+0000", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7010000000000008e-06 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128780+0000", 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2024-09-16T22:16:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128825+0000", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5179000000000002e-05 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128908+0000", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3034999999999994e-05 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129097+0000", 2024-09-16T22:16:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018963500000000001 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.132248+0000", 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031504860000000001 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135120+0000", 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028716150000000001 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135178+0000", 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8621e-05 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135224+0000", 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5593999999999997e-05 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1412 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'8, mlcod=28'6)", 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.288337313, 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049201369999999998, 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179547+0000", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000004e-07 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179551+0000", 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3819999999999997e-06 2024-09-16T22:16:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179562+0000", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0567999999999999e-05 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179575+0000", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3492000000000001e-05 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179614+0000", 2024-09-16T22:16:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8284999999999997e-05 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179627+0000", 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3618000000000001e-05 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184435+0000", 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048080299999999996 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184466+0000", 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1232999999999997e-05 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1796 3.b 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head [write 760672~25760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.679073+0000", 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7888100859999998, 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039611480000000003, 2024-09-16T22:16:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1796 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679073+0000", 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679073+0000", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679075+0000", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.181e-06 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679123+0000", 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8111000000000001e-05 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679135+0000", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2247e-05 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679141+0000", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5090000000000001e-06 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679217+0000", 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6302000000000002e-05 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679268+0000", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1425999999999999e-05 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679349+0000", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0607999999999998e-05 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.679884+0000", 2024-09-16T22:16:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053554399999999999 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.682960+0000", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030756389999999998 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.682996+0000", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6072000000000003e-05 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.683035+0000", 2024-09-16T22:16:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8509e-05 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1888 3.b 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795097+0000", 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6727863739999993, 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039699669999999996, 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1888 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795097+0000", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795097+0000", 2024-09-16T22:16:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795099+0000", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2979999999999999e-06 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795103+0000", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795114+0000", 2024-09-16T22:16:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1192999999999999e-05 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795119+0000", 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4079999999999997e-06 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795182+0000", 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2904999999999995e-05 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795269+0000", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6580000000000001e-05 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795399+0000", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013043000000000001 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795901+0000", 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000501452 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.798971+0000", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030702500000000001 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.799026+0000", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4687e-05 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.799067+0000", 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1391999999999999e-05 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1894 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'12, mlcod=28'10)", 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6721695889999992, 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055310519999999998, 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795715+0000", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6700000000000003e-07 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795720+0000", 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-16T22:16:32.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795730+0000", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9250000000000007e-06 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795736+0000", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6500000000000001e-06 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795782+0000", 2024-09-16T22:16:32.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6669999999999999e-05 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795801+0000", 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8202e-05 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801216+0000", 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054153120000000003 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801245+0000", 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9122e-05 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13220 3.b 3:d58c95a5:::rbd_data.11cdc482f8dc.000000000000013f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.3614569750000003, 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049084810000000001, 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13220 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1699999999999997e-07 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106432+0000", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106442+0000", 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0137e-05 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106447+0000", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8949999999999997e-06 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106495+0000", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8322000000000003e-05 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106569+0000", 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4054999999999995e-05 2024-09-16T22:16:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106749+0000", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180483 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108197+0000", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014480020000000001 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111252+0000", 2024-09-16T22:16:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030545139999999999 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111303+0000", 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1406999999999998e-05 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111335+0000", 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.167e-05 2024-09-16T22:16:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13369 3.b 3:d34300fb:::rbd_data.11cdc482f8dc.0000000000000188:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.30256887, 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005364906, 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13369 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165316+0000", 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.47e-07 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165320+0000", 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-06 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165330+0000", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0448999999999999e-05 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165337+0000", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165380+0000", 2024-09-16T22:16:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2908000000000003e-05 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165446+0000", 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5754999999999996e-05 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165546+0000", 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100691 2024-09-16T22:16:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170501+0000", 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049542609999999997 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170589+0000", 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8287000000000004e-05 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170632+0000", 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985999999999998e-05 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170680+0000", 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7799000000000001e-05 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13370 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.3023959569999999, 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005335607, 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13370 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.92e-07 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165491+0000", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165498+0000", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.708e-06 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165502+0000", 2024-09-16T22:16:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1130000000000003e-06 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165731+0000", 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000228936 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165812+0000", 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1353000000000005e-05 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170823+0000", 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050107490000000001 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13378 3.0 e31/23 3:06bf120b:::rbd_data.11cdc482f8dc.000000000000018b:head v 31'30, mlcod=31'28)", 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2968132279999995, 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005199207, 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171071+0000", 2024-09-16T22:16:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171075+0000", 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171087+0000", 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1708e-05 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171092+0000", 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1460000000000004e-06 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171135+0000", 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2388999999999997e-05 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171152+0000", 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708999999999999e-05 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176231+0000", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050785659999999996 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176270+0000", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8766999999999997e-05 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13380 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2966867950000003, 2024-09-16T22:16:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005174423, 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13380 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171198+0000", 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1500000000000003e-07 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171201+0000", 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7809999999999999e-06 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171220+0000", 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9000000000000001e-05 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171229+0000", 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1149999999999994e-06 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171277+0000", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8062000000000002e-05 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171353+0000", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5986000000000001e-05 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171555+0000", 2024-09-16T22:16:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202418 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.173540+0000", 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001984675 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176272+0000", 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027319890000000002 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176317+0000", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4798e-05 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176371+0000", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4684e-05 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13381 3.6 e31/23 3:6539fbfd:::rbd_data.11cdc482f8dc.000000000000018e:head v 31'101, mlcod=31'99)", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2963769220000003, 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047825259999999996, 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3499999999999996e-07 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171512+0000", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0950000000000003e-06 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171528+0000", 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781999999999999e-05 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171535+0000", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171580+0000", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4663999999999998e-05 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171591+0000", 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1026000000000001e-05 2024-09-16T22:16:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176254+0000", 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046628090000000004 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176289+0000", 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5383999999999999e-05 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13383 3.8 e31/23 3:129ab390:::rbd_data.11cdc482f8dc.0000000000000191:head v 31'100, mlcod=31'98)", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.293737041, 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050429230000000004, 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174151+0000", 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1489999999999996e-06 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174161+0000", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4380000000000001e-06 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174165+0000", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2359999999999998e-06 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174208+0000", 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2738000000000003e-05 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174225+0000", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7788e-05 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179148+0000", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049220849999999997 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179189+0000", 2024-09-16T22:16:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1909000000000001e-05 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13390 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2912776380000004, 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058595619999999996, 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13390 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176607+0000", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9800000000000003e-07 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176610+0000", 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6030000000000001e-06 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176620+0000", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999995e-06 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176625+0000", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-06 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176669+0000", 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3532000000000003e-05 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176744+0000", 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5045999999999999e-05 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182466+0000", 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005721673 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13397 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2851278720000003, 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005279087, 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13397 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182757+0000", 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1299999999999998e-07 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182761+0000", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0709999999999996e-06 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182769+0000", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6589999999999996e-06 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182776+0000", 2024-09-16T22:16:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182822+0000", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970999999999997e-05 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182827+0000", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185510+0000", 2024-09-16T22:16:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026825009999999999 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185598+0000", 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.831e-05 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185814+0000", 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021624099999999999 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186536+0000", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072142400000000002 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187969+0000", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014327859999999999 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187998+0000", 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9713000000000002e-05 2024-09-16T22:16:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.188035+0000", 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6516000000000001e-05 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.c epoch 32/23 rep_tid 368 entries 32'37 (0'0) error 3:3694cf9a:::rbd_data.11cdc482f8dc.00000000000001a1:head by client.4557.0:13410 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'35)", 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2732025379999996, 2024-09-16T22:16:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043492569999999996, 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.48e-07 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194685+0000", 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:16:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194699+0000", 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3246e-05 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194710+0000", 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1351e-05 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194764+0000", 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3566000000000002e-05 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.199030+0000", 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042666659999999997 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13438 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.261655814, 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053131510000000003, 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13438 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206229+0000", 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-06 2024-09-16T22:16:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206235+0000", 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.237e-06 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206248+0000", 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3192e-05 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206254+0000", 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8159999999999999e-06 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206294+0000", 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0533e-05 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206378+0000", 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4405000000000002e-05 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211541+0000", 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051624430000000001 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13453 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2560232019999997, 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050623450000000002, 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13453 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211861+0000", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0070000000000001e-06 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211867+0000", 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1440000000000002e-06 2024-09-16T22:16:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211880+0000", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3453e-05 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211889+0000", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0089999999999996e-06 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211939+0000", 2024-09-16T22:16:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9904999999999997e-05 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212026+0000", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7007000000000002e-05 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212200+0000", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017373700000000001 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216837+0000", 2024-09-16T22:16:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046371540000000001 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216867+0000", 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0380999999999999e-05 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216898+0000", 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0754999999999999e-05 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216923+0000", 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4793000000000001e-05 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13461 3.0 e32/23 3:033943f4:::rbd_data.11cdc482f8dc.00000000000001b7:head v 32'34, mlcod=32'32)", 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.215384+0000", 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2524998929999995, 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040495560000000002, 2024-09-16T22:16:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215384+0000", 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215384+0000", 2024-09-16T22:16:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215384+0000", 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215390+0000", 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215403+0000", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3315999999999999e-05 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215410+0000", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7139999999999996e-06 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215456+0000", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5822999999999998e-05 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.215474+0000", 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7884e-05 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219395+0000", 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003921206 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219433+0000", 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8492999999999999e-05 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13469 3.b 3:d0ddb2a3:::rbd_data.11cdc482f8dc.00000000000001be:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2481768090000003, 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054339640000000003, 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13469 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219708+0000", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.009e-06 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219712+0000", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219723+0000", 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1141999999999999e-05 2024-09-16T22:16:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219730+0000", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1210000000000001e-06 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219790+0000", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9917999999999998e-05 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219924+0000", 2024-09-16T22:16:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133807 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225141+0000", 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052165809999999996 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13602 3.b 3:d1d109b3:::rbd_data.11cdc482f8dc.00000000000001fe:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.1945324680000002, 2024-09-16T22:16:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050799549999999997, 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13602 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273352+0000", 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999995e-07 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273356+0000", 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273365+0000", 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9709999999999993e-06 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273371+0000", 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273409+0000", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7790000000000002e-05 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273457+0000", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7754999999999999e-05 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273562+0000", 2024-09-16T22:16:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104851 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275934+0000", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023726329999999999 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278377+0000", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024423790000000002 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278410+0000", 2024-09-16T22:16:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3056999999999999e-05 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278431+0000", 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1231000000000001e-05 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:32.571 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:32.572 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:32.573 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:32.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:32.575 INFO:teuthology.orchestra.run.smithi031.stdout:1196 writesame 0xea4a752 thru 0xea54b67 data_size 0x3(0xa416 bytes) 2024-09-16T22:16:32.575 INFO:teuthology.orchestra.run.smithi031.stdout:1197 write 0x1964fdc thru 0x196ac5f (0x5c84 bytes) 2024-09-16T22:16:32.575 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:32.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:32.576 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:32.576 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:32.576 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:32.576 INFO:teuthology.orchestra.run.smithi031.stdout:1199 trunc from 0xea54b68 to 0x429960b 2024-09-16T22:16:32.577 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:32.577 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:32.577 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:32.574+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:32.584 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:32.605 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:32.605 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:32.626 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:32.626 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:32.635 INFO:teuthology.orchestra.run.smithi031.stdout:1200 writesame 0xc7c6efb thru 0xc7cee1f data_size 0xb(0x7f25 bytes) 2024-09-16T22:16:32.635 INFO:teuthology.orchestra.run.smithi031.stdout:1201 write 0x66b2863 thru 0x66bb39d (0x8b3b bytes) 2024-09-16T22:16:32.652 INFO:teuthology.orchestra.run.smithi031.stdout:1202 trunc from 0xc7cee20 to 0x12dade7 2024-09-16T22:16:32.673 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:32.675+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:32.712 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:32.712 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:32.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:32.802 INFO:teuthology.orchestra.run.smithi031.stdout:1203 punch from 0x80e331 to 0x81bff3, (0xdcc2 bytes) 2024-09-16T22:16:32.802 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:32.804+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:32.808 INFO:teuthology.orchestra.run.smithi031.stdout:1204 trunc from 0x12dade7 to 0x59de94e 2024-09-16T22:16:32.813 INFO:teuthology.orchestra.run.smithi031.stdout:1206 punch from 0x2b072e8 to 0x2b0bc56, (0x496e bytes) 2024-09-16T22:16:32.813 INFO:teuthology.orchestra.run.smithi031.stdout:1207 read 0x4beba45 thru 0x4becb2b (0x10e7 bytes) 2024-09-16T22:16:32.825 INFO:teuthology.orchestra.run.smithi031.stdout:1208 write 0x9c55830 thru 0x9c5b269 (0x5a3a bytes) 2024-09-16T22:16:32.837 INFO:teuthology.orchestra.run.smithi031.stdout:1209 write 0xb69e07d thru 0xb6a17bb (0x373f bytes) 2024-09-16T22:16:32.855 INFO:teuthology.orchestra.run.smithi031.stdout:1210 trunc from 0xb6a17bc to 0x27068f7 2024-09-16T22:16:32.865 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:32.875 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:32.877+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:32.883 INFO:teuthology.orchestra.run.smithi031.stdout:1213 write 0xcc6f222 thru 0xcc72016 (0x2df5 bytes) 2024-09-16T22:16:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:1214 punch from 0x32d8b to 0x3a2f5, (0x756a bytes) 2024-09-16T22:16:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:1216 trunc from 0xcc72017 to 0xc26ba61 2024-09-16T22:16:32.897 INFO:teuthology.orchestra.run.smithi031.stdout:1217 write 0x91c58a1 thru 0x91d3a36 (0xe196 bytes) 2024-09-16T22:16:32.898 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:32.898 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:32.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:32.909 INFO:teuthology.orchestra.run.smithi031.stdout:1220 write 0xcb27c10 thru 0xcb33919 (0xbd0a bytes) 2024-09-16T22:16:32.916 INFO:teuthology.orchestra.run.smithi031.stdout:1221 punch from 0x3d2b744 to 0x3d33a5a, (0x8316 bytes) 2024-09-16T22:16:32.917 INFO:teuthology.orchestra.run.smithi031.stdout:1222 clone 9 order 22 su 131072 sc 14 2024-09-16T22:16:32.976 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:32.978+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:33.023 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.023 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:33.116 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:33.118+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:33.178 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:33.179+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:33.180 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.180 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:33.279 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:33.280+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:33.335 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.336 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:33.385 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:33.387+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone8 from 0xcb3391a (overlap 0x12dade7) to 0x3c55fb 2024-09-16T22:16:33.481 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:33.482+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:33.499 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.499 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:33.550 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T22:16:33.558 INFO:teuthology.orchestra.run.smithi031.stdout:1223 punch from 0x113d14d to 0x114a937, (0xd7ea bytes) 2024-09-16T22:16:33.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.562 INFO:teuthology.orchestra.run.smithi031.stdout:1225 write 0x2141615 thru 0x2149c34 (0x8620 bytes) 2024-09-16T22:16:33.572 INFO:teuthology.orchestra.run.smithi031.stdout:1226 read 0x64d9ebe thru 0x64da840 (0x983 bytes) 2024-09-16T22:16:33.572 INFO:teuthology.orchestra.run.smithi031.stdout:1227 read 0x14446cf thru 0x1451f04 (0xd836 bytes) 2024-09-16T22:16:33.572 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.572 INFO:teuthology.orchestra.run.smithi031.stdout:1229 read 0x44aff50 thru 0x44b2984 (0x2a35 bytes) 2024-09-16T22:16:33.572 INFO:teuthology.orchestra.run.smithi031.stdout:1230 writesame 0x2812ea6 thru 0x281f849 data_size 0x74(0xc9a4 bytes) 2024-09-16T22:16:33.581 INFO:teuthology.orchestra.run.smithi031.stdout:1231 write 0x4d03112 thru 0x4d1061a (0xd509 bytes) 2024-09-16T22:16:33.581 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:33.583+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:33.585 INFO:teuthology.orchestra.run.smithi031.stdout:1232 punch from 0x198657 to 0x198c11, (0x5ba bytes) 2024-09-16T22:16:33.585 INFO:teuthology.orchestra.run.smithi031.stdout:1235 write 0xc6de2dc thru 0xc6e61ea (0x7f0f bytes) 2024-09-16T22:16:33.594 INFO:teuthology.orchestra.run.smithi031.stdout:1236 write 0xcd7d7dc thru 0xcd84d7a (0x759f bytes) 2024-09-16T22:16:33.599 INFO:teuthology.orchestra.run.smithi031.stdout:1237 read 0x6597744 thru 0x6599dfd (0x26ba bytes) 2024-09-16T22:16:33.599 INFO:teuthology.orchestra.run.smithi031.stdout:1238 write 0x22e1f4d thru 0x22e5b6c (0x3c20 bytes) 2024-09-16T22:16:33.613 INFO:teuthology.orchestra.run.smithi031.stdout:1239 write 0xd316e80 thru 0xd326aa3 (0xfc24 bytes) 2024-09-16T22:16:33.615 INFO:teuthology.orchestra.run.smithi031.stdout:1240 write 0x9f2299d thru 0x9f28cf3 (0x6357 bytes) 2024-09-16T22:16:33.630 INFO:teuthology.orchestra.run.smithi031.stdout:1241 trunc from 0xd326aa4 to 0x57aba82 2024-09-16T22:16:33.633 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.633 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:33.664 INFO:teuthology.orchestra.run.smithi031.stdout:1242 writesame 0xb151203 thru 0xb152198 data_size 0x13(0xf96 bytes) 2024-09-16T22:16:33.664 INFO:teuthology.orchestra.run.smithi031.stdout:1243 read 0x367a828 thru 0x367f989 (0x5162 bytes) 2024-09-16T22:16:33.665 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.665 INFO:teuthology.orchestra.run.smithi031.stdout:1246 punch from 0x45ae095 to 0x45bbabe, (0xda29 bytes) 2024-09-16T22:16:33.665 INFO:teuthology.orchestra.run.smithi031.stdout:1247 punch from 0x6562252 to 0x6568b13, (0x68c1 bytes) 2024-09-16T22:16:33.665 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:1252 write 0xb5279db thru 0xb530979 (0x8f9f bytes) 2024-09-16T22:16:33.672 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.672 INFO:teuthology.orchestra.run.smithi031.stdout:1255 punch from 0x742e661 to 0x743a15c, (0xbafb bytes) 2024-09-16T22:16:33.672 INFO:teuthology.orchestra.run.smithi031.stdout:1256 read 0x27122aa thru 0x271a94c (0x86a3 bytes) 2024-09-16T22:16:33.672 INFO:teuthology.orchestra.run.smithi031.stdout:1257 writesame 0x1944a61 thru 0x1948a9e data_size 0x3(0x403e bytes) 2024-09-16T22:16:33.683 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:33.684+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:33.683 INFO:teuthology.orchestra.run.smithi031.stdout:1258 punch from 0x7c7f05d to 0x7c8c401, (0xd3a4 bytes) 2024-09-16T22:16:33.683 INFO:teuthology.orchestra.run.smithi031.stdout:1259 read 0xf9ceb9 thru 0xfaa729 (0xd871 bytes) 2024-09-16T22:16:33.684 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.684 INFO:teuthology.orchestra.run.smithi031.stdout:1261 writesame 0x6b28b1f thru 0x6b2d0cf data_size 0x13(0x45b1 bytes) 2024-09-16T22:16:33.685 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.692 INFO:teuthology.orchestra.run.smithi031.stdout:1264 writesame 0xcaeb370 thru 0xcaeec67 data_size 0x4(0x38f8 bytes) 2024-09-16T22:16:33.692 INFO:teuthology.orchestra.run.smithi031.stdout:1265 write 0x6287b8e thru 0x6293736 (0xbba9 bytes) 2024-09-16T22:16:33.705 INFO:teuthology.orchestra.run.smithi031.stdout:1267 trunc from 0xcaeec68 to 0x2aa6bd5 2024-09-16T22:16:33.783 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:33.785+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:33.795 INFO:teuthology.orchestra.run.smithi031.stdout:1269 punch from 0x1e4abd9 to 0x1e4f142, (0x4569 bytes) 2024-09-16T22:16:33.804 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.804 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.804 INFO:teuthology.orchestra.run.smithi031.stdout:1270 write 0x605f82c thru 0x60642c7 (0x4a9c bytes) 2024-09-16T22:16:33.810 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:33.810 INFO:teuthology.orchestra.run.smithi031.stdout:1272 clone 10 order 21 su 16384 sc 5 2024-09-16T22:16:33.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:33.884 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:33.886+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:33.960 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:33.960 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:33.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:34.012 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:34.014+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:34.075 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.075 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:34.085 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:34.087+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:34.187 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:34.188+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:34.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:16:34.235 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.235 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:34.288 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:34.289+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:34.300 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone9 from 0x60642c8 (overlap 0x2aa6bd5) to 0x19973e6 2024-09-16T22:16:34.389 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:34.391+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:34.409 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.409 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.412 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T22:16:34.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:34.434 INFO:teuthology.orchestra.run.smithi031.stdout:1274 write 0x812b05c thru 0x8137eeb (0xce90 bytes) 2024-09-16T22:16:34.444 INFO:teuthology.orchestra.run.smithi031.stdout:1276 punch from 0x653f103 to 0x654e3e8, (0xf2e5 bytes) 2024-09-16T22:16:34.445 INFO:teuthology.orchestra.run.smithi031.stdout:1277 read 0x6394972 thru 0x639f461 (0xaaf0 bytes) 2024-09-16T22:16:34.445 INFO:teuthology.orchestra.run.smithi031.stdout:1278 write 0x341d38d thru 0x34294c6 (0xc13a bytes) 2024-09-16T22:16:34.465 INFO:teuthology.orchestra.run.smithi031.stdout:1279 write 0x28caad0 thru 0x28cfaff (0x5030 bytes) 2024-09-16T22:16:34.484 INFO:teuthology.orchestra.run.smithi031.stdout:1280 trunc from 0x8137eec to 0x553aeb1 2024-09-16T22:16:34.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:34.491+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:34.506 INFO:teuthology.orchestra.run.smithi031.stdout:1282 read 0x2382b03 thru 0x238fe82 (0xd380 bytes) 2024-09-16T22:16:34.508 INFO:teuthology.orchestra.run.smithi031.stdout:1283 trunc from 0x553aeb1 to 0x5f6cdf4 2024-09-16T22:16:34.514 INFO:teuthology.orchestra.run.smithi031.stdout:1284 writesame 0x24d75e0 thru 0x24dfed5 data_size 0x2f(0x88f6 bytes) 2024-09-16T22:16:34.544 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.545 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.546 INFO:teuthology.orchestra.run.smithi031.stdout:1286 read 0x29d256b thru 0x29d8de5 (0x687b bytes) 2024-09-16T22:16:34.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:34.554 INFO:teuthology.orchestra.run.smithi031.stdout:1288 trunc from 0x5f6cdf4 to 0xc497581 2024-09-16T22:16:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:1289 read 0x198125d thru 0x198dde5 (0xcb89 bytes) 2024-09-16T22:16:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:1292 write 0xf6432e thru 0xf6a530 (0x6203 bytes) 2024-09-16T22:16:34.579 INFO:teuthology.orchestra.run.smithi031.stdout:1293 write 0x44690c1 thru 0x447212c (0x906c bytes) 2024-09-16T22:16:34.589 INFO:teuthology.orchestra.run.smithi031.stdout:1294 punch from 0x3449640 to 0x344f035, (0x59f5 bytes) 2024-09-16T22:16:34.589 INFO:teuthology.orchestra.run.smithi031.stdout:1296 write 0xabda9c7 thru 0xabdea09 (0x4043 bytes) 2024-09-16T22:16:34.590 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:34.591+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:34.596 INFO:teuthology.orchestra.run.smithi031.stdout:1297 write 0x1cb312 thru 0x1d6263 (0xaf52 bytes) 2024-09-16T22:16:34.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:34.653 INFO:teuthology.orchestra.run.smithi031.stdout:1298 writesame 0xdd18148 thru 0xdd1e62d data_size 0x7b(0x64e6 bytes) 2024-09-16T22:16:34.654 INFO:teuthology.orchestra.run.smithi031.stdout:1301 punch from 0x3305d4e to 0x330669e, (0x950 bytes) 2024-09-16T22:16:34.654 INFO:teuthology.orchestra.run.smithi031.stdout:1303 punch from 0x62e0765 to 0x62e1364, (0xbff bytes) 2024-09-16T22:16:34.669 INFO:teuthology.orchestra.run.smithi031.stdout:1305 trunc from 0xdd1e62e to 0x24930b 2024-09-16T22:16:34.678 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.678 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:34.690 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:34.692+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:34.714 INFO:teuthology.orchestra.run.smithi031.stdout:1306 write 0xb0f5e0b thru 0xb0f96da (0x38d0 bytes) 2024-09-16T22:16:34.717 INFO:teuthology.orchestra.run.smithi031.stdout:1307 read 0x729b52b thru 0x72a7f8c (0xca62 bytes) 2024-09-16T22:16:34.717 INFO:teuthology.orchestra.run.smithi031.stdout:1308 read 0x7ee346e thru 0x7ee4bfc (0x178f bytes) 2024-09-16T22:16:34.717 INFO:teuthology.orchestra.run.smithi031.stdout:1310 read 0x67f8e7 thru 0x68e3aa (0xeac4 bytes) 2024-09-16T22:16:34.718 INFO:teuthology.orchestra.run.smithi031.stdout:1312 read 0x167e3fd thru 0x1681b53 (0x3757 bytes) 2024-09-16T22:16:34.718 INFO:teuthology.orchestra.run.smithi031.stdout:1313 flatten 2024-09-16T22:16:34.733 INFO:teuthology.orchestra.run.smithi031.stdout:1314 writesame 0xd2db99d thru 0xd2e2586 data_size 0x13(0x6bea bytes) 2024-09-16T22:16:34.733 INFO:teuthology.orchestra.run.smithi031.stdout:1315 punch from 0x2dcd747 to 0x2dd98c5, (0xc17e bytes) 2024-09-16T22:16:34.733 INFO:teuthology.orchestra.run.smithi031.stdout:1316 write 0xafb9f96 thru 0xafc097b (0x69e6 bytes) 2024-09-16T22:16:34.736 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:34.736 INFO:teuthology.orchestra.run.smithi031.stdout:1318 read 0x9dc3b44 thru 0x9dc7e52 (0x430f bytes) 2024-09-16T22:16:34.737 INFO:teuthology.orchestra.run.smithi031.stdout:1319 clone 11 order 19 su 8192 sc 11 2024-09-16T22:16:34.780 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:34.780 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:34.792 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:34.793+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:34.795 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:34.795 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:34.797 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:34.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:34.814 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:34.830 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:34.830 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:34.847 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:34.847 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:34.863 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:34.880 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:34.892 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:34.894+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:34.896 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:34.897 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:34.946 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:34.947 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:34.947 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:34.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:34.993 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:34.995+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:35.059 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:35.059 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:35.111 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:35.113+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:35.171 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18012 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6721472500000001, 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026593847, 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18012 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500539+0000", 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1399999999999995e-07 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500542+0000", 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9110000000000002e-06 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500544+0000", 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3130000000000001e-06 2024-09-16T22:16:35.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527056+0000", 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026511349999999999 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527095+0000", 2024-09-16T22:16:35.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9530000000000003e-05 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527132+0000", 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6613999999999999e-05 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18013 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6721352889999999, 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026682926999999999, 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18013 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3799999999999999e-07 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500551+0000", 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8100000000000001e-07 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500560+0000", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4010000000000003e-06 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500567+0000", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4019999999999999e-06 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527155+0000", 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026588284 2024-09-16T22:16:35.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527195+0000", 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.019e-05 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527233+0000", 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7431000000000001e-05 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18014 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6721144250000002, 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026763540999999998, 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18014 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:35.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500572+0000", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.085e-06 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500576+0000", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8229999999999997e-06 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500578+0000", 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3360000000000002e-06 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527256+0000", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026677421 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527297+0000", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1028000000000001e-05 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527334+0000", 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7469000000000003e-05 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18015 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14123008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6721040399999998, 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026853495000000002, 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18015 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500582+0000", 2024-09-16T22:16:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500585+0000", 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9749999999999999e-06 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500587+0000", 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527357+0000", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026769826 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527398+0000", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0329e-05 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527435+0000", 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6968999999999998e-05 2024-09-16T22:16:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18016 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14204928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.67209495, 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026943981999999998, 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18016 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500591+0000", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500594+0000", 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500597+0000", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527457+0000", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026860288 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527497+0000", 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9490999999999998e-05 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527534+0000", 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18017 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.672085381, 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027035519000000001, 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18017 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500601+0000", 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2200000000000003e-07 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500604+0000", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.959e-06 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500606+0000", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527557+0000", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026950941999999999 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527599+0000", 2024-09-16T22:16:35.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1962000000000002e-05 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527635+0000", 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18018 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14368768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6720763600000002, 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027127043, 2024-09-16T22:16:35.199 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18018 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500610+0000", 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500613+0000", 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.473e-06 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500616+0000", 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2699999999999999e-06 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527659+0000", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027043754 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527699+0000", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9847e-05 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527736+0000", 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6610999999999999e-05 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18019 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14450688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6720657910000001, 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027226185, 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18019 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500620+0000", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500623+0000", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.687e-06 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500627+0000", 2024-09-16T22:16:35.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3220000000000001e-06 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500630+0000", 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.146e-06 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527758+0000", 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027128635000000002 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527799+0000", 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1009e-05 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527846+0000", 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6173e-05 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18020 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:35.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6720524700000001, 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027315971000000001, 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18020 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500634+0000", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500638+0000", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8e-06 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500640+0000", 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527869+0000", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027229410999999999 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527911+0000", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1402000000000001e-05 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527949+0000", 2024-09-16T22:16:35.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7901000000000002e-05 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18021 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6720424390000002, 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027409273000000001, 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18021 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.23e-07 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500644+0000", 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3900000000000003e-07 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500647+0000", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500649+0000", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527972+0000", 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027322470000000001 2024-09-16T22:16:35.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528013+0000", 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1211000000000002e-05 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528052+0000", 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9184000000000003e-05 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18022 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14696448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.672030388, 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027498129, 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18022 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500656+0000", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999999e-07 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500659+0000", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2150000000000001e-06 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500665+0000", 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:16:35.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528075+0000", 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027410336 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528116+0000", 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1366999999999999e-05 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528153+0000", 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6640999999999997e-05 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18023 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14778368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6720173799999998, 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027586167000000002, 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.222 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18023 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500669+0000", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7900000000000007e-07 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500673+0000", 2024-09-16T22:16:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3200000000000001e-06 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500676+0000", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528176+0000", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027500419000000002 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528217+0000", 2024-09-16T22:16:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893999999999998e-05 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528254+0000", 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6874e-05 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18024 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14860288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6719980890000001, 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665443000000001, 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18024 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2e-07 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500689+0000", 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.207e-06 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500692+0000", 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2780000000000002e-06 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500694+0000", 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528276+0000", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027581847 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528317+0000", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0417999999999998e-05 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528352+0000", 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5879e-05 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18025 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6719877639999998, 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02775803, 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18025 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500698+0000", 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:35.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500699+0000", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500701+0000", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500707+0000", 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:35.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528375+0000", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027668663 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528418+0000", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2073000000000002e-05 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528455+0000", 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7880000000000003e-05 2024-09-16T22:16:35.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18026 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15024128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.67197526, 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027845890000000002, 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18026 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500711+0000", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7499999999999998e-07 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500714+0000", 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500717+0000", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528478+0000", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027761082999999999 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528519+0000", 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1090000000000001e-05 2024-09-16T22:16:35.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528556+0000", 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7058999999999999e-05 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18027 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15106048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6719659189999998, 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027937167999999998, 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18027 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500720+0000", 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500723+0000", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500725+0000", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528578+0000", 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027852946999999999 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528619+0000", 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1069000000000003e-05 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528656+0000", 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6958000000000001e-05 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18028 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15187968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.671956615, 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028028412999999999, 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18028 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:35.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500792+0000", 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3039000000000004e-05 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500793+0000", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.734e-06 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500798+0000", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3579999999999996e-06 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500800+0000", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8289999999999999e-06 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528680+0000", 2024-09-16T22:16:35.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027879095 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528718+0000", 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8627000000000001e-05 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528757+0000", 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8730999999999999e-05 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18029 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6718796120000001, 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028059634999999999, 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18029 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500975+0000", 2024-09-16T22:16:35.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016965599999999999 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500977+0000", 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500983+0000", 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2100000000000001e-06 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500986+0000", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1360000000000001e-06 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528779+0000", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027793629 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528820+0000", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0689999999999998e-05 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528865+0000", 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5145000000000002e-05 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18030 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6716935419999999, 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027977591, 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18030 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501041+0000", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9799000000000002e-05 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501047+0000", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0009999999999998e-06 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501052+0000", 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-16T22:16:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501055+0000", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528890+0000", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027835642000000001 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528930+0000", 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0076000000000001e-05 2024-09-16T22:16:35.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528969+0000", 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8741000000000001e-05 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18031 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15433728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.6716252200000001, 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028011306999999999, 2024-09-16T22:16:35.256 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18031 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505887+0000", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048273159999999999 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505891+0000", 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.36e-06 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505897+0000", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7440000000000001e-06 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505908+0000", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0421000000000001e-05 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528993+0000", 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023084957 2024-09-16T22:16:35.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529034+0000", 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1023999999999999e-05 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529071+0000", 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7484999999999998e-05 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.260 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:16:35.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:35.263 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:35.263+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:35.363 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:35.364+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:23572 3.12 3:4b9312dd:::rbd_header.11cd82e047b7:head [call rbd.snapshot_get in=23b,call rbd.parent_overlap_get in=29b,call rbd.get_flags in=20b,call rbd.get_protection_status in=32b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.411071+0000", 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.000138858, 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015644199999999999, 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23572 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411071+0000", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411071+0000", 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411071+0000", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2300000000000001e-07 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411074+0000", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411080+0000", 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5979999999999996e-06 2024-09-16T22:16:35.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411083+0000", 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4809999999999998e-06 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411118+0000", 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5091e-05 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.411194+0000", 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5514999999999998e-05 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-16T22:16:35.413 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:35.424 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T22:16:35.463 INFO:teuthology.orchestra.run.smithi031.stdout:1321 punch from 0x2d9f036 to 0x2dab41d, (0xc3e7 bytes) 2024-09-16T22:16:35.464 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:35.466+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:35.468 INFO:teuthology.orchestra.run.smithi031.stdout:1323 punch from 0x41cf18c to 0x41ded1c, (0xfb90 bytes) 2024-09-16T22:16:35.468 INFO:teuthology.orchestra.run.smithi031.stdout:1324 read 0x8572b21 thru 0x857de35 (0xb315 bytes) 2024-09-16T22:16:35.468 INFO:teuthology.orchestra.run.smithi031.stdout:1325 punch from 0x4bc9362 to 0x4bcda43, (0x46e1 bytes) 2024-09-16T22:16:35.468 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.468 INFO:teuthology.orchestra.run.smithi031.stdout:1328 writesame 0x2c82d8c thru 0x2c902c5 data_size 0x31(0xd53a bytes) 2024-09-16T22:16:35.480 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.480 INFO:teuthology.orchestra.run.smithi031.stdout:1330 write 0x667a6ed thru 0x6689985 (0xf299 bytes) 2024-09-16T22:16:35.490 INFO:teuthology.orchestra.run.smithi031.stdout:1331 write 0xd1d65e5 thru 0xd1dca47 (0x6463 bytes) 2024-09-16T22:16:35.501 INFO:teuthology.orchestra.run.smithi031.stdout:1332 read 0x199883a thru 0x199d3fc (0x4bc3 bytes) 2024-09-16T22:16:35.501 INFO:teuthology.orchestra.run.smithi031.stdout:1333 write 0x516dbc4 thru 0x5176fb2 (0x93ef bytes) 2024-09-16T22:16:35.512 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.512 INFO:teuthology.orchestra.run.smithi031.stdout:1337 write 0xa5c499b thru 0xa5cb1fb (0x6861 bytes) 2024-09-16T22:16:35.529 INFO:teuthology.orchestra.run.smithi031.stdout:1338 trunc from 0xd2e2587 to 0x82005b2 2024-09-16T22:16:35.530 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.530 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:35.530 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:35.530 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:35.530 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:35.559 INFO:teuthology.orchestra.run.smithi031.stdout:1339 trunc from 0x82005b2 to 0xabdd711 2024-09-16T22:16:35.566 INFO:teuthology.orchestra.run.smithi031.stdout:1340 read 0x9da6279 thru 0x9da6f91 (0xd19 bytes) 2024-09-16T22:16:35.566 INFO:teuthology.orchestra.run.smithi031.stdout:1341 punch from 0x7ba5bf to 0x7c481f, (0xa260 bytes) 2024-09-16T22:16:35.566 INFO:teuthology.orchestra.run.smithi031.stdout:1344 read 0x9f7cda1 thru 0x9f8223d (0x549d bytes) 2024-09-16T22:16:35.566 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.566 INFO:teuthology.orchestra.run.smithi031.stdout:1346 write 0x25ace01 thru 0x25b27ba (0x59ba bytes) 2024-09-16T22:16:35.574 INFO:teuthology.orchestra.run.smithi031.stdout:1348 read 0x7b73106 thru 0x7b76ab8 (0x39b3 bytes) 2024-09-16T22:16:35.574 INFO:teuthology.orchestra.run.smithi031.stdout:1349 writesame 0x606675c thru 0x606a1e1 data_size 0x21(0x3a86 bytes) 2024-09-16T22:16:35.582 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:35.584+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:35.587 INFO:teuthology.orchestra.run.smithi031.stdout:1350 trunc from 0xabdd711 to 0x33c3bfd 2024-09-16T22:16:35.618 INFO:teuthology.orchestra.run.smithi031.stdout:1352 read 0xaf2bad thru 0xaf9960 (0x6db4 bytes) 2024-09-16T22:16:35.627 INFO:teuthology.orchestra.run.smithi031.stdout:1353 write 0x60e1081 thru 0x60ee228 (0xd1a8 bytes) 2024-09-16T22:16:35.634 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.642 INFO:teuthology.orchestra.run.smithi031.stdout:1356 write 0x862a96b thru 0x8637c46 (0xd2dc bytes) 2024-09-16T22:16:35.650 INFO:teuthology.orchestra.run.smithi031.stdout:1358 punch from 0x4011c82 to 0x4017477, (0x57f5 bytes) 2024-09-16T22:16:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:1359 write 0xb87bd17 thru 0xb881ed0 (0x61ba bytes) 2024-09-16T22:16:35.663 INFO:teuthology.orchestra.run.smithi031.stdout:1360 read 0x34187a8 thru 0x341ff53 (0x77ac bytes) 2024-09-16T22:16:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:1361 punch from 0x7def09e to 0x7dfa94b, (0xb8ad bytes) 2024-09-16T22:16:35.665 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:35.667+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:35.674 INFO:teuthology.orchestra.run.smithi031.stdout:1363 trunc from 0xb881ed1 to 0x306f2ec 2024-09-16T22:16:35.700 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1373 3.b e28/23 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head v 28'7, mlcod=27'5)", 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.572277751, 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057964999999999996, 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9700000000000003e-07 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129215+0000", 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8830000000000002e-06 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129222+0000", 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8469999999999998e-06 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129227+0000", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4870000000000004e-06 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129279+0000", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2219000000000003e-05 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129299+0000", 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9877999999999998e-05 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.134962+0000", 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056633079999999997 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135009+0000", 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6581e-05 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.108709503, 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:16:35.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:16:35.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1818 3.1 3:8772e73b:::rbd_data.11cd70c2eac3.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.980952504999999, 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062717169999999996, 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1818 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720539+0000", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720543+0000", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.464e-06 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720552+0000", 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3090000000000001e-06 2024-09-16T22:16:35.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720560+0000", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6909999999999997e-06 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720604+0000", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3838999999999998e-05 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720683+0000", 2024-09-16T22:16:35.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8916000000000001e-05 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.721544+0000", 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086141500000000003 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726702+0000", 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005157671 2024-09-16T22:16:35.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726720+0000", 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717999999999999e-05 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726768+0000", 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8035999999999999e-05 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726809+0000", 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1539000000000003e-05 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1933 3.12 3:4b562b7a:::rbd_data.11cd70c2eac3.000000000000000d:head [truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.836399525999999, 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056569769999999997, 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1933 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865092+0000", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6309999999999999e-06 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865097+0000", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865116+0000", 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8766e-05 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865124+0000", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0099999999999995e-06 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865179+0000", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5169e-05 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865257+0000", 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8221999999999997e-05 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865377+0000", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119473 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.867211+0000", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00183428 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870680+0000", 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003468563 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870710+0000", 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0337000000000001e-05 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870747+0000", 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7228000000000003e-05 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9870 3.1 3:8781002f:::rbd_data.11cdc482f8dc.00000000000000b6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8963713169999998, 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057426480000000004, 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9870 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805127+0000", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6860000000000006e-06 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805137+0000", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7000000000000003e-06 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805144+0000", 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-09-16T22:16:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805215+0000", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1155999999999994e-05 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805261+0000", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6279000000000002e-05 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805365+0000", 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103929 2024-09-16T22:16:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805721+0000", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035559099999999999 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810764+0000", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050430689999999999 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810817+0000", 2024-09-16T22:16:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3029999999999999e-05 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810861+0000", 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4372999999999997e-05 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:12107 3.4 3:202b7b59:::rbd_data.11cdc482f8dc.0000000000000116:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7016989579999997, 2024-09-16T22:16:35.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054483370000000001, 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12107 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999792+0000", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9300000000000006e-07 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999796+0000", 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4279999999999998e-06 2024-09-16T22:16:35.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999807+0000", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0824e-05 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999813+0000", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9789999999999998e-06 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999863+0000", 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0127000000000003e-05 2024-09-16T22:16:35.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999976+0000", 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112828 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.005239+0000", 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052631580000000004 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13077 3.4 3:26c5fb8d:::rbd_data.11cdc482f8dc.0000000000000133:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6067007849999992, 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053770570000000002, 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13077 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094789+0000", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094790+0000", 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7499999999999999e-07 2024-09-16T22:16:35.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094794+0000", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.907e-06 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094806+0000", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.221e-05 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094813+0000", 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1969999999999999e-06 2024-09-16T22:16:35.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094864+0000", 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1103000000000002e-05 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.094911+0000", 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7077999999999999e-05 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.095014+0000", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102525 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100050+0000", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050360680000000003 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100075+0000", 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4459000000000001e-05 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100134+0000", 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9725000000000002e-05 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.100166+0000", 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1909999999999998e-05 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13225 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5930036160000007, 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056928450000000002, 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13225 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108487+0000", 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6899999999999996e-07 2024-09-16T22:16:35.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108498+0000", 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0733999999999999e-05 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108509+0000", 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553999999999999e-05 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108515+0000", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9480000000000002e-06 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108574+0000", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8906000000000001e-05 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108713+0000", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013846 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114179+0000", 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054662740000000001 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13240 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5869525079999995, 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057273300000000001, 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13240 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114538+0000", 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114542+0000", 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2710000000000003e-06 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114552+0000", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6029999999999994e-06 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114564+0000", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1555e-05 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114611+0000", 2024-09-16T22:16:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7686999999999999e-05 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114691+0000", 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9938000000000006e-05 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114862+0000", 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017119999999999999 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.119890+0000", 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050279849999999996 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120170+0000", 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027961400000000002 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120224+0000", 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3922000000000003e-05 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120265+0000", 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0760000000000003e-05 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13339 3.12 3:4fd424ff:::rbd_data.11cdc482f8dc.0000000000000179:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5508357260000007, 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065394440000000002, 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13339 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150656+0000", 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150665+0000", 2024-09-16T22:16:35.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8529999999999998e-06 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150685+0000", 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0222999999999999e-05 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150697+0000", 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150774+0000", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6954999999999998e-05 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150917+0000", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014322900000000001 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151076+0000", 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015856799999999999 2024-09-16T22:16:35.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151448+0000", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037228599999999999 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157138+0000", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056900379999999997 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157169+0000", 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0910000000000001e-05 2024-09-16T22:16:35.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157194+0000", 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4542999999999999e-05 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13353 3.1 3:80996581:::rbd_data.11cdc482f8dc.0000000000000185:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5433151580000004, 2024-09-16T22:16:35.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054098480000000001, 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13353 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:35.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158176+0000", 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158180+0000", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158189+0000", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9190000000000005e-06 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158196+0000", 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8340000000000001e-06 2024-09-16T22:16:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158243+0000", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7157999999999998e-05 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158319+0000", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5574000000000005e-05 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.163585+0000", 2024-09-16T22:16:35.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052660169999999996 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13391 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.524837754, 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066380959999999996, 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13391 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176653+0000", 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.305e-06 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176658+0000", 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176670+0000", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1982e-05 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176675+0000", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1220000000000003e-06 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176719+0000", 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4036000000000003e-05 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176798+0000", 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8981999999999997e-05 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183290+0000", 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064919390000000004 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5178019549999995, 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:16:35.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:16:35.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13466 3.12 3:49b34a3f:::rbd_data.11cdc482f8dc.00000000000001bc:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4820740699999995, 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055848399999999998, 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13466 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219419+0000", 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6860000000000002e-06 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219426+0000", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0480000000000001e-06 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219431+0000", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219939+0000", 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050803999999999997 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.220051+0000", 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011102700000000001 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225001+0000", 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049501299999999996 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4672872659999996, 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.453961176, 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:16:35.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:16:35.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13595 3.5 e32/23 3:a7f0f76d:::rbd_data.11cdc482f8dc.00000000000001fd:head v 32'67, mlcod=32'65)", 2024-09-16T22:16:35.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4309947639999994, 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053849240000000001, 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270496+0000", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270500+0000", 2024-09-16T22:16:35.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0350000000000003e-06 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270508+0000", 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4970000000000001e-06 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270513+0000", 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7720000000000002e-06 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270549+0000", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5667000000000003e-05 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270567+0000", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8507000000000001e-05 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275827+0000", 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005260069 2024-09-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275880+0000", 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2831000000000003e-05 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1124626639999997, 2024-09-16T22:16:35.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:35.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:16:35.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:35.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4557.0:16275 3.4 e35/23 ondisk, result = 0)", 2024-09-16T22:16:35.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.0958801510000002, 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084889960000000004, 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605611+0000", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3009999999999999e-06 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605616+0000", 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:35.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605631+0000", 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4827000000000001e-05 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605640+0000", 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3840000000000003e-06 2024-09-16T22:16:35.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614068+0000", 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084278679999999998 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614081+0000", 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2576e-05 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614099+0000", 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8108e-05 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:17626 3.12 3:49f8983c:::rbd_data.11cd82e047b7.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3080192~273] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.2469078109999998, 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057507920000000002, 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:35.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 17626 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454584+0000", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7400000000000001e-06 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456196+0000", 2024-09-16T22:16:35.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001612523 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456213+0000", 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6266999999999999e-05 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456221+0000", 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.456e-06 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:35.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456302+0000", 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1427999999999994e-05 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456399+0000", 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6055000000000001e-05 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456573+0000", 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017492300000000001 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.459141+0000", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002567747 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460269+0000", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00112813 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460304+0000", 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4857000000000002e-05 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:35.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460333+0000", 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8666e-05 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:35.810 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1364 write 0xe6b2da0 thru 0xe6b8a9a (0x5cfb bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1365 write 0xa6f2505 thru 0xa6f6856 (0x4352 bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1366 write 0x99b46e0 thru 0x99bf178 (0xaa99 bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1368 write 0xbe2c035 thru 0xbe3872f (0xc6fb bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1369 read 0x9611816 thru 0x9618bab (0x7396 bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1370 read 0xb6e8900 thru 0xb6f0a57 (0x8158 bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1371 read 0xc1fc7c1 thru 0xc201614 (0x4e54 bytes) 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1373 trunc from 0xe6b8a9b to 0x6af0009 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.814 INFO:teuthology.orchestra.run.smithi031.stdout:1375 read 0x28e5a66 thru 0x28effe9 (0xa584 bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1377 read 0x2b906b9 thru 0x2b9c273 (0xbbbb bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1380 write 0x84adfaf thru 0x84b38e0 (0x5932 bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1381 read 0xd646f7 thru 0xd71ed0 (0xd7da bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1384 writesame 0x9a57059 thru 0x9a591b5 data_size 0x3(0x215d bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1386 read 0x1a2771e thru 0x1a27790 (0x73 bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1387 read 0x12408b6 thru 0x1250611 (0xfd5c bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1388 punch from 0x7470bcc to 0x747379e, (0x2bd2 bytes) 2024-09-16T22:16:35.815 INFO:teuthology.orchestra.run.smithi031.stdout:1390 write 0x9097a3 thru 0x912094 (0x88f2 bytes) 2024-09-16T22:16:35.815 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:35.813+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:35.818 INFO:teuthology.orchestra.run.smithi031.stdout:1391 write 0x7e532c2 thru 0x7e5ddda (0xab19 bytes) 2024-09-16T22:16:35.825 INFO:teuthology.orchestra.run.smithi031.stdout:1392 write 0x7980d57 thru 0x7982cdb (0x1f85 bytes) 2024-09-16T22:16:35.827 INFO:teuthology.orchestra.run.smithi031.stdout:1395 clone 12 order 18 su 8192 sc 13 2024-09-16T22:16:35.912 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:35.914+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:35.956 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:35.956 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:35.956 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:35.956 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:35.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:36.013 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:36.014+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:36.068 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:36.068 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:36.068 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:36.068 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:36.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:36.120 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:36.122+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:36.181 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1373 3.b 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.054166167, 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064652499999999996, 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1373 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0299999999999998e-07 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128763+0000", 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-06 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128773+0000", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7010000000000008e-06 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128780+0000", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128825+0000", 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5179000000000002e-05 2024-09-16T22:16:36.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128908+0000", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3034999999999994e-05 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129097+0000", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018963500000000001 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.132248+0000", 2024-09-16T22:16:36.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031504860000000001 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135120+0000", 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028716150000000001 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135178+0000", 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8621e-05 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135224+0000", 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5593999999999997e-05 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1412 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'8, mlcod=28'6)", 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.00337845, 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049201369999999998, 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179547+0000", 2024-09-16T22:16:36.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000004e-07 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179551+0000", 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3819999999999997e-06 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179562+0000", 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0567999999999999e-05 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179575+0000", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3492000000000001e-05 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179614+0000", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8284999999999997e-05 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179627+0000", 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3618000000000001e-05 2024-09-16T22:16:36.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184435+0000", 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048080299999999996 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184466+0000", 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1232999999999997e-05 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1894 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'12, mlcod=28'10)", 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.387210725999999, 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055310519999999998, 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795715+0000", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6700000000000003e-07 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795720+0000", 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795730+0000", 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9250000000000007e-06 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795736+0000", 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6500000000000001e-06 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795782+0000", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6669999999999999e-05 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795801+0000", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8202e-05 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801216+0000", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054153120000000003 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801245+0000", 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9122e-05 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13220 3.b 3:d58c95a5:::rbd_data.11cdc482f8dc.000000000000013f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.076498111999999, 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049084810000000001, 2024-09-16T22:16:36.197 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13220 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:36.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1699999999999997e-07 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106432+0000", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106442+0000", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0137e-05 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106447+0000", 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8949999999999997e-06 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106495+0000", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8322000000000003e-05 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106569+0000", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4054999999999995e-05 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106749+0000", 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180483 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108197+0000", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014480020000000001 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111252+0000", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030545139999999999 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111303+0000", 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1406999999999998e-05 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111335+0000", 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.167e-05 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13369 3.b 3:d34300fb:::rbd_data.11cdc482f8dc.0000000000000188:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.017610007, 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005364906, 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13369 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165316+0000", 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.47e-07 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165320+0000", 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-06 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165330+0000", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0448999999999999e-05 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165337+0000", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165380+0000", 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2908000000000003e-05 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165446+0000", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5754999999999996e-05 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165546+0000", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100691 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170501+0000", 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049542609999999997 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170589+0000", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8287000000000004e-05 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170632+0000", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985999999999998e-05 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170680+0000", 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7799000000000001e-05 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13370 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.017437094, 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005335607, 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13370 2024-09-16T22:16:36.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.92e-07 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165491+0000", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165498+0000", 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.708e-06 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165502+0000", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1130000000000003e-06 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165731+0000", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000228936 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165812+0000", 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1353000000000005e-05 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170823+0000", 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050107490000000001 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13378 3.0 e31/23 3:06bf120b:::rbd_data.11cdc482f8dc.000000000000018b:head v 31'30, mlcod=31'28)", 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.011854365, 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005199207, 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171071+0000", 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171075+0000", 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171087+0000", 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1708e-05 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171092+0000", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1460000000000004e-06 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171135+0000", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2388999999999997e-05 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171152+0000", 2024-09-16T22:16:36.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708999999999999e-05 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176231+0000", 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050785659999999996 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176270+0000", 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8766999999999997e-05 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13380 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.011727931999999, 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005174423, 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13380 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171198+0000", 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1500000000000003e-07 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171201+0000", 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7809999999999999e-06 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171220+0000", 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9000000000000001e-05 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171229+0000", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1149999999999994e-06 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171277+0000", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8062000000000002e-05 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171353+0000", 2024-09-16T22:16:36.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5986000000000001e-05 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171555+0000", 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202418 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.173540+0000", 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001984675 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176272+0000", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027319890000000002 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176317+0000", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4798e-05 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176371+0000", 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4684e-05 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13381 3.6 e31/23 3:6539fbfd:::rbd_data.11cdc482f8dc.000000000000018e:head v 31'101, mlcod=31'99)", 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.011418059, 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047825259999999996, 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3499999999999996e-07 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171512+0000", 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0950000000000003e-06 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171528+0000", 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781999999999999e-05 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171535+0000", 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171580+0000", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4663999999999998e-05 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171591+0000", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1026000000000001e-05 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176254+0000", 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046628090000000004 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176289+0000", 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5383999999999999e-05 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13383 3.8 e31/23 3:129ab390:::rbd_data.11cdc482f8dc.0000000000000191:head v 31'100, mlcod=31'98)", 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.008778178, 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050429230000000004, 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174151+0000", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1489999999999996e-06 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174161+0000", 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4380000000000001e-06 2024-09-16T22:16:36.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174165+0000", 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2359999999999998e-06 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174208+0000", 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2738000000000003e-05 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174225+0000", 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7788e-05 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179148+0000", 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049220849999999997 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179189+0000", 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1909000000000001e-05 2024-09-16T22:16:36.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13390 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.006318775, 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058595619999999996, 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13390 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176607+0000", 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9800000000000003e-07 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176610+0000", 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6030000000000001e-06 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176620+0000", 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999995e-06 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176625+0000", 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-06 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176669+0000", 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3532000000000003e-05 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176744+0000", 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5045999999999999e-05 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182466+0000", 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005721673 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13397 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.000169009, 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005279087, 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13397 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182757+0000", 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1299999999999998e-07 2024-09-16T22:16:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182761+0000", 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0709999999999996e-06 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182769+0000", 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6589999999999996e-06 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182776+0000", 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182822+0000", 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970999999999997e-05 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182827+0000", 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185510+0000", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026825009999999999 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185598+0000", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.831e-05 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185814+0000", 2024-09-16T22:16:36.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021624099999999999 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186536+0000", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072142400000000002 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187969+0000", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014327859999999999 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187998+0000", 2024-09-16T22:16:36.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9713000000000002e-05 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.188035+0000", 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6516000000000001e-05 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.c epoch 32/23 rep_tid 368 entries 32'37 (0'0) error 3:3694cf9a:::rbd_data.11cdc482f8dc.00000000000001a1:head by client.4557.0:13410 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'35)", 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9882436749999997, 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043492569999999996, 2024-09-16T22:16:36.244 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.48e-07 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194685+0000", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194699+0000", 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3246e-05 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194710+0000", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1351e-05 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194764+0000", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3566000000000002e-05 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.199030+0000", 2024-09-16T22:16:36.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042666659999999997 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13438 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9766969509999992, 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053131510000000003, 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13438 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206229+0000", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-06 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206235+0000", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.237e-06 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206248+0000", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3192e-05 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206254+0000", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8159999999999999e-06 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206294+0000", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0533e-05 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206378+0000", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4405000000000002e-05 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211541+0000", 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051624430000000001 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13453 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9710643389999998, 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050623450000000002, 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13453 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211861+0000", 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0070000000000001e-06 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211867+0000", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1440000000000002e-06 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211880+0000", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3453e-05 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211889+0000", 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0089999999999996e-06 2024-09-16T22:16:36.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211939+0000", 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9904999999999997e-05 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212026+0000", 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7007000000000002e-05 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212200+0000", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017373700000000001 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216837+0000", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046371540000000001 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216867+0000", 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0380999999999999e-05 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216898+0000", 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0754999999999999e-05 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216923+0000", 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4793000000000001e-05 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13469 3.b 3:d0ddb2a3:::rbd_data.11cdc482f8dc.00000000000001be:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.259 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9632179460000003, 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054339640000000003, 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13469 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219708+0000", 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.009e-06 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219712+0000", 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219723+0000", 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1141999999999999e-05 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219730+0000", 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1210000000000001e-06 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219790+0000", 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9917999999999998e-05 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219924+0000", 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133807 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225141+0000", 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052165809999999996 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13602 3.b 3:d1d109b3:::rbd_data.11cdc482f8dc.00000000000001fe:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9095736050000003, 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050799549999999997, 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13602 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273352+0000", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999995e-07 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273356+0000", 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-09-16T22:16:36.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273365+0000", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9709999999999993e-06 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273371+0000", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273409+0000", 2024-09-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7790000000000002e-05 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273457+0000", 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7754999999999999e-05 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273562+0000", 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104851 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275934+0000", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023726329999999999 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278377+0000", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024423790000000002 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278410+0000", 2024-09-16T22:16:36.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3056999999999999e-05 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278431+0000", 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1231000000000001e-05 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:23604 3.b 3:d4792668:::rbd_data.11cd3076d18c.0000000000000003:head [sparse-read 0~1441792] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.749864274, 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046028680000000004, 2024-09-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23604 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433061+0000", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8299999999999998e-07 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433063+0000", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9410000000000001e-06 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433067+0000", 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4750000000000004e-06 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433072+0000", 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1410000000000004e-06 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.435649+0000", 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025763689999999998 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.435698+0000", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9119000000000002e-05 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.437663+0000", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019654400000000002 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:23725 3.6 e39/23 3:6036e720:::rbd_data.11cd7b8f383d.0000000000000100:head v 39'330, mlcod=38'328)", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:36.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63676609100000003, 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047562580000000002, 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0599999999999999e-07 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546165+0000", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546175+0000", 2024-09-16T22:16:36.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0081000000000001e-05 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546182+0000", 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9099999999999999e-06 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546224+0000", 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2358999999999999e-05 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546241+0000", 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6614000000000001e-05 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550873+0000", 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046324189999999996 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550915+0000", 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1708000000000001e-05 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:23726 3.0 e39/23 3:034d15c2:::rbd_data.11cd7b8f383d.0000000000000101:head v 39'50, mlcod=38'48)", 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63656767199999997, 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045589699999999999, 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546358+0000", 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546363+0000", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7130000000000002e-06 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546373+0000", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0019e-05 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546380+0000", 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-16T22:16:36.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546430+0000", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9805000000000002e-05 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546450+0000", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550885+0000", 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044349719999999997 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:36.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550916+0000", 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1211000000000003e-05 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:36.284 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone11 intact 2024-09-16T22:16:36.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:36.285 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:36.286+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:36.386 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:36.387+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:36.391 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T22:16:36.400 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:36.401 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:36.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:36.486 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:36.488+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:36.551 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:36.552 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:36.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:36.605 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:36.607+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:36.622 INFO:teuthology.orchestra.run.smithi031.stdout:1396 read 0x101120c thru 0x1011666 (0x45b bytes) 2024-09-16T22:16:36.632 INFO:teuthology.orchestra.run.smithi031.stdout:1400 writesame 0xa98cede thru 0xa9903e8 data_size 0x1(0x350b bytes) 2024-09-16T22:16:36.632 INFO:teuthology.orchestra.run.smithi031.stdout:1401 read 0x9b0ece2 thru 0x9b1b562 (0xc881 bytes) 2024-09-16T22:16:36.638 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:36.639 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:36.639 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T22:16:36.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T22:16:36.640 INFO:teuthology.orchestra.run.smithi031.stdout:1402 writesame 0xcb5e521 thru 0xcb61053 data_size 0x1(0x2b33 bytes) 2024-09-16T22:16:36.640 INFO:teuthology.orchestra.run.smithi031.stdout:1403 read 0x3098323 thru 0x309bd5d (0x3a3b bytes) 2024-09-16T22:16:36.642 INFO:teuthology.orchestra.run.smithi031.stdout:1404 write 0x47195bf thru 0x471f2af (0x5cf1 bytes) 2024-09-16T22:16:36.651 INFO:teuthology.orchestra.run.smithi031.stdout:1405 clone 13 order 24 su 65536 sc 2 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:782 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.616882+0000", 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.060560039, 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081481000000000001, 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 782 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616882+0000", 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616882+0000", 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616883+0000", 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8000000000000005e-07 2024-09-16T22:16:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616888+0000", 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7350000000000004e-06 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616898+0000", 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8910000000000007e-06 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.616906+0000", 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5310000000000001e-06 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619230+0000", 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023232460000000002 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619322+0000", 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2019000000000001e-05 2024-09-16T22:16:36.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619705+0000", 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038361599999999998 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622144+0000", 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024388890000000001 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624926+0000", 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002781791 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624982+0000", 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5711000000000002e-05 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625030+0000", 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8891000000000003e-05 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:785 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617134+0000", 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.060307969, 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007993389, 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 785 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617134+0000", 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617134+0000", 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617135+0000", 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6700000000000002e-07 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617140+0000", 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5480000000000003e-06 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617151+0000", 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1449e-05 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617158+0000", 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.620147+0000", 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029890569999999998 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.620205+0000", 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8085999999999999e-05 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.620425+0000", 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000220473 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622181+0000", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017559539999999999 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625041+0000", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028595220000000002 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625089+0000", 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.761e-05 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625128+0000", 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9283999999999999e-05 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:791 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.059838640000001, 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011713016999999999, 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 791 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:36.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617605+0000", 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.78e-07 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617610+0000", 2024-09-16T22:16:36.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617625+0000", 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5088e-05 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617632+0000", 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1960000000000002e-06 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622247+0000", 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004614613 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622294+0000", 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7210999999999999e-05 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622548+0000", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025380800000000001 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624807+0000", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022596249999999999 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629226+0000", 2024-09-16T22:16:36.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044184940000000002 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629278+0000", 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2487000000000001e-05 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629317+0000", 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8454999999999997e-05 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:792 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.059669287, 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011640707, 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 792 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617774+0000", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6499999999999999e-07 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617778+0000", 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.656e-06 2024-09-16T22:16:36.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617789+0000", 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0428000000000001e-05 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617795+0000", 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.981e-06 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622907+0000", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051125449999999996 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622970+0000", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2992999999999998e-05 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623170+0000", 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000199252 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624855+0000", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001685256 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629324+0000", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044690859999999997 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629363+0000", 2024-09-16T22:16:36.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9209999999999999e-05 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629414+0000", 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0634999999999999e-05 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:793 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.059143882000001, 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011223963999999999, 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 793 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618300+0000", 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-06 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618306+0000", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618318+0000", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1590000000000001e-05 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618326+0000", 2024-09-16T22:16:36.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6200000000000005e-06 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623487+0000", 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051610579999999996 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623539+0000", 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1709999999999998e-05 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623742+0000", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020280100000000001 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624907+0000", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011651509999999999 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629432+0000", 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045250020000000002 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629481+0000", 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9163999999999999e-05 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629522+0000", 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1486000000000002e-05 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:794 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.059032341, 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011215777999999999, 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 794 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618411+0000", 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0589999999999999e-06 2024-09-16T22:16:36.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618417+0000", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618429+0000", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1627000000000001e-05 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618436+0000", 2024-09-16T22:16:36.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4749999999999996e-06 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624022+0000", 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055863629999999996 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624075+0000", 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2540999999999997e-05 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624472+0000", 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039755500000000001 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626817+0000", 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023447849999999998 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629532+0000", 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027150970000000001 2024-09-16T22:16:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629584+0000", 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1338999999999998e-05 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629626+0000", 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2168e-05 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:797 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.058833630000001, 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011182566999999999, 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 797 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618610+0000", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3e-07 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618614+0000", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4120000000000003e-06 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618624+0000", 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0051000000000001e-05 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618640+0000", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6376999999999999e-05 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625267+0000", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066267929999999997 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625332+0000", 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4689000000000004e-05 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625554+0000", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000222029 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629641+0000", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040872929999999997 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629697+0000", 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5637000000000003e-05 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629751+0000", 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3980000000000002e-05 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629791+0000", 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0575999999999999e-05 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:798 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.058601777, 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011102747, 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 798 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1900000000000002e-07 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618847+0000", 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1680000000000003e-06 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618858+0000", 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1868000000000001e-05 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618865+0000", 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625906+0000", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070405290000000002 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625998+0000", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2559999999999995e-05 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626397+0000", 2024-09-16T22:16:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039847599999999998 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629651+0000", 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032540680000000002 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629842+0000", 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019086799999999999 2024-09-16T22:16:36.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629899+0000", 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7666e-05 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629943+0000", 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4106000000000001e-05 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:800 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.058301272, 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011059087, 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 800 2024-09-16T22:16:36.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619142+0000", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4700000000000001e-07 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619153+0000", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0783e-05 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619164+0000", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1338999999999999e-05 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619171+0000", 2024-09-16T22:16:36.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.579e-06 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626861+0000", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076901560000000001 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626956+0000", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4984999999999999e-05 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627511+0000", 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055479699999999999 2024-09-16T22:16:36.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629995+0000", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024841479999999998 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630109+0000", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011454 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630160+0000", 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0371999999999998e-05 2024-09-16T22:16:36.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630200+0000", 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0640999999999999e-05 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:804 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.05806965, 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010918458000000001, 2024-09-16T22:16:36.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 804 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619374+0000", 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619380+0000", 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619393+0000", 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2612999999999999e-05 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619400+0000", 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627970+0000", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085695540000000001 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628019+0000", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9518999999999998e-05 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628226+0000", 2024-09-16T22:16:36.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020676000000000001 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630042+0000", 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001816493 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630211+0000", 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016820600000000001 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630253+0000", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2281999999999999e-05 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630291+0000", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8433999999999998e-05 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:805 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:36.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.057715211, 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010651252, 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 805 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:36.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619728+0000", 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0400000000000005e-07 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619733+0000", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7829999999999998e-06 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619743+0000", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7799999999999995e-06 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619749+0000", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5699999999999998e-06 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628524+0000", 2024-09-16T22:16:36.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087750190000000002 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628601+0000", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6953000000000007e-05 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628948+0000", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034638399999999998 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630098+0000", 2024-09-16T22:16:36.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011506140000000001 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630302+0000", 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020387500000000001 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630343+0000", 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0839e-05 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630378+0000", 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5531000000000003e-05 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1369 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.126007+0000", 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.55143563, 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091844779999999994, 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1369 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126007+0000", 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126007+0000", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126007+0000", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126010+0000", 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126016+0000", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4699999999999999e-06 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126023+0000", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0519999999999998e-06 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.129275+0000", 2024-09-16T22:16:36.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032527900000000002 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.129364+0000", 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8857000000000007e-05 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.129713+0000", 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034894400000000001 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.131203+0000", 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014895830000000001 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135132+0000", 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003929241 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135162+0000", 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0066999999999999e-05 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135191+0000", 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9261999999999999e-05 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1370 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.551408567999999, 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092874129999999996, 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1370 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126038+0000", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126045+0000", 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126053+0000", 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5240000000000003e-06 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130267+0000", 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042138549999999999 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130362+0000", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.522e-05 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130774+0000", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041218299999999999 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.131499+0000", 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072460999999999999 2024-09-16T22:16:36.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135246+0000", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037469640000000002 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135284+0000", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8572000000000003e-05 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135321+0000", 2024-09-16T22:16:36.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6847999999999997e-05 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:6556 3.f 3:f21b5a5c:::rbd_data.11cdc482f8dc.0000000000000047:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.497031+0000", 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.180411169999999, 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084996659999999995, 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6556 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497031+0000", 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497031+0000", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497033+0000", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4270000000000001e-06 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497038+0000", 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6559999999999997e-06 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497054+0000", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5707000000000001e-05 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497063+0000", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7479999999999999e-06 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497122+0000", 2024-09-16T22:16:36.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9287999999999999e-05 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.497247+0000", 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012463200000000001 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.505531+0000", 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082842079999999995 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13327 3.3 3:c2effc6c:::rbd_data.11cdc482f8dc.0000000000000171:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145117+0000", 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.532325762999999, 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091040040000000006, 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13327 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145117+0000", 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145117+0000", 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145117+0000", 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8100000000000002e-07 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145122+0000", 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.267e-06 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145132+0000", 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0244000000000001e-05 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145137+0000", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.916e-06 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145181+0000", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4268999999999999e-05 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145256+0000", 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4684999999999994e-05 2024-09-16T22:16:36.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145426+0000", 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000170657 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.146396+0000", 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096991500000000001 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154126+0000", 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077301130000000003 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154183+0000", 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6418000000000001e-05 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154221+0000", 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7738999999999999e-05 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13328 3.3 3:c2e7e3ca:::rbd_data.11cdc482f8dc.0000000000000177:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.532280849999999, 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091967899999999998, 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13328 2024-09-16T22:16:36.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9599999999999999e-07 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145165+0000", 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.208e-06 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145173+0000", 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6529999999999994e-06 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145179+0000", 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145702+0000", 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052302200000000005 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145820+0000", 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118544 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154358+0000", 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008538159 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.1b epoch 31/23 rep_tid 366 entries 31'49 (0'0) error 3:df7903cc:::rbd_data.11cdc482f8dc.0000000000000178:head by client.4557.0:13329 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'47)", 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145635+0000", 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.531807403, 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008608938, 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145635+0000", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145635+0000", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145636+0000", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145643+0000", 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3660000000000003e-06 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145660+0000", 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6957e-05 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145673+0000", 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3366999999999999e-05 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145733+0000", 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9802000000000001e-05 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154244+0000", 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085106750000000005 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:13330 3.13 e31/23 3:cb153b9d:::rbd_data.11cdc482f8dc.0000000000000172:head v 31'47, mlcod=31'45)", 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145772+0000", 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.531670074999999, 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085034859999999993, 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145772+0000", 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145772+0000", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145773+0000", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8500000000000001e-07 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145777+0000", 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4490000000000001e-06 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145786+0000", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4179999999999997e-06 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145804+0000", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8408999999999999e-05 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145878+0000", 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3697999999999998e-05 2024-09-16T22:16:36.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145889+0000", 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1026000000000001e-05 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154245+0000", 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083565010000000006 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154276+0000", 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.04e-05 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13334 3.15 3:a9fa41bc:::rbd_data.11cdc482f8dc.000000000000017b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.529994428, 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094895459999999997, 2024-09-16T22:16:36.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13334 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147449+0000", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147454+0000", 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6319999999999999e-06 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3657e-05 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147473+0000", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9800000000000003e-06 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147523+0000", 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9397999999999997e-05 2024-09-16T22:16:36.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147645+0000", 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012283699999999999 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.156938+0000", 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009292079 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13395 3.f 3:f350c6c8:::rbd_data.11cdc482f8dc.0000000000000194:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.4966328, 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014172446, 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13395 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:36.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180811+0000", 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9500000000000001e-07 2024-09-16T22:16:36.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180815+0000", 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1400000000000002e-06 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180825+0000", 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:36.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180830+0000", 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5350000000000004e-06 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180870+0000", 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9391000000000003e-05 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180873+0000", 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6380000000000002e-06 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186724+0000", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058504869999999997 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186813+0000", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9585000000000004e-05 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.187027+0000", 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021384099999999999 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.189456+0000", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024287449999999999 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194872+0000", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054157670000000002 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194949+0000", 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7781000000000006e-05 2024-09-16T22:16:36.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194982+0000", 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2669999999999997e-05 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:36.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:36.794 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:36.795+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:36.894 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:36.896+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:36.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 40, 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-16T22:15:44.319460+0000", 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-16T22:16:36.223779+0000", 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-16T22:15:56.374983+0000", 2024-09-16T22:16:36.942 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-16T22:15:46.438697+0000", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-16T22:16:36.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:15:56.583719+0000", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:36.944 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-16T22:16:36.945 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:36.946 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:36.947 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:36.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-16T22:16:36.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:16:36.950 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:02.234097+0000", 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:36.951 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "22", 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:36.952 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 22, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:36.953 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:36.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-16T22:16:36.955 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:13.789572+0000", 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:36.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-16T22:16:36.957 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.958 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "40", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 14, 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 40, 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:36.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:36.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:36.961 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:36.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "44de8de0-e16a-4325-94b9-1cc61f4829db", 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.963 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.965 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.966 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2585124124", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2585124124", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2585124124", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2585124124", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0e05fbd1-9707-40b8-91a3-198c6f5b7865", 2024-09-16T22:16:36.968 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T22:16:36.969 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.972 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3271285223", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3271285223", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3271285223", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3271285223", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.973 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "b080bc29-0a92-4200-ae3f-0a17a1722145", 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.9312591552734375, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.974 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.977 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2874892035", 2024-09-16T22:16:36.978 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2874892035", 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2874892035", 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2874892035", 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0bc64c75-486c-40ac-abaf-381e151623bd", 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.979 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3768411773", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3768411773", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3768411773", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3768411773", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fae627d1-6bc2-4e5b-85e1-334e5e05f830", 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.985 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6824", 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6825", 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.986 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6826", 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6827", 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6830", 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6831", 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6828", 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6829", 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6825/3421901083", 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6827/3421901083", 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3421901083", 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6829/3421901083", 2024-09-16T22:16:36.990 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "989a79cd-ef84-4bf3-a64f-9d101d0b29d7", 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.992 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6816", 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6817", 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6818", 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.994 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6819", 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6822", 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6823", 2024-09-16T22:16:36.995 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6820", 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6821", 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:36.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6817/335833224", 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6819/335833224", 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6823/335833224", 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6821/335833224", 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.997 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "92f145b7-e868-43fd-9abe-68c43a407ae2", 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6808", 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6809", 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:36.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6810", 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6811", 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6814", 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6815", 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.001 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6812", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6813", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6809/1592419018", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6811/1592419018", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1592419018", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1592419018", 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:37.002 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "373f8002-53a4-4ca7-bbbf-70e8d56b3571", 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6800", 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6801", 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6802", 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6803", 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.005 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6806", 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6807", 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:37.006 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6804", 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6805", 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6801/3448871901", 2024-09-16T22:16:37.007 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6803/3448871901", 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3448871901", 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3448871901", 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.008 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.033837+0000", 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.799099+0000", 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.769970+0000", 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.010 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.965029+0000", 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.706036+0000", 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:37.011 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.158188+0000", 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.012 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.722931+0000", 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.485009+0000", 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-16T22:16:37.013 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-16T22:16:37.014 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-16T22:16:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:37.016 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:37.017 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:37.017 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:37.017 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:37.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:37.019 INFO:tasks.thrashosds.thrasher:j is {'epoch': 40, 'fsid': '57fe7da5-e356-48a4-ba33-5558b1026a32', 'created': '2024-09-16T22:15:44.319460+0000', 'modified': '2024-09-16T22:16:36.223779+0000', 'last_up_change': '2024-09-16T22:15:56.374983+0000', 'last_in_change': '2024-09-16T22:15:46.438697+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T22:15:56.583719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T22:16:02.234097+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T22:16:13.789572+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 40, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '44de8de0-e16a-4325-94b9-1cc61f4829db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2585124124}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2585124124}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2585124124}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2585124124}]}, 'public_addr': '172.21.15.31:6817/2585124124', 'cluster_addr': '172.21.15.31:6819/2585124124', 'heartbeat_back_addr': '172.21.15.31:6823/2585124124', 'heartbeat_front_addr': '172.21.15.31:6821/2585124124', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3271285223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3271285223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3271285223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3271285223}]}, 'public_addr': '172.21.15.31:6801/3271285223', 'cluster_addr': '172.21.15.31:6803/3271285223', 'heartbeat_back_addr': '172.21.15.31:6807/3271285223', 'heartbeat_front_addr': '172.21.15.31:6805/3271285223', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b080bc29-0a92-4200-ae3f-0a17a1722145', 'up': 1, 'in': 1, 'weight': 0.9312591552734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2874892035}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2874892035}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2874892035}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2874892035}]}, 'public_addr': '172.21.15.31:6809/2874892035', 'cluster_addr': '172.21.15.31:6811/2874892035', 'heartbeat_back_addr': '172.21.15.31:6815/2874892035', 'heartbeat_front_addr': '172.21.15.31:6813/2874892035', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0bc64c75-486c-40ac-abaf-381e151623bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3768411773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3768411773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3768411773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3768411773}]}, 'public_addr': '172.21.15.31:6825/3768411773', 'cluster_addr': '172.21.15.31:6827/3768411773', 'heartbeat_back_addr': '172.21.15.31:6831/3768411773', 'heartbeat_front_addr': '172.21.15.31:6829/3768411773', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 3421901083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3421901083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3421901083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3421901083}]}, 'public_addr': '172.21.15.92:6825/3421901083', 'cluster_addr': '172.21.15.92:6827/3421901083', 'heartbeat_back_addr': '172.21.15.92:6831/3421901083', 'heartbeat_front_addr': '172.21.15.92:6829/3421901083', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 335833224}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 335833224}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 335833224}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 335833224}]}, 'public_addr': '172.21.15.92:6817/335833224', 'cluster_addr': '172.21.15.92:6819/335833224', 'heartbeat_back_addr': '172.21.15.92:6823/335833224', 'heartbeat_front_addr': '172.21.15.92:6821/335833224', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '92f145b7-e868-43fd-9abe-68c43a407ae2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1592419018}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1592419018}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1592419018}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1592419018}]}, 'public_addr': '172.21.15.92:6809/1592419018', 'cluster_addr': '172.21.15.92:6811/1592419018', 'heartbeat_back_addr': '172.21.15.92:6815/1592419018', 'heartbeat_front_addr': '172.21.15.92:6813/1592419018', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '373f8002-53a4-4ca7-bbbf-70e8d56b3571', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3448871901}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3448871901}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3448871901}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3448871901}]}, 'public_addr': '172.21.15.92:6801/3448871901', 'cluster_addr': '172.21.15.92:6803/3448871901', 'heartbeat_back_addr': '172.21.15.92:6807/3448871901', 'heartbeat_front_addr': '172.21.15.92:6805/3448871901', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.033837+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.799099+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.769970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.965029+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.706036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.158188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.722931+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.485009+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T22:16:37.019 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T22:16:37.019 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:37.018+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:37.033 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:37.034 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:37.051 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:37.068 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:37.084 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:37.085 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:37.102 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:37.117 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:37.119+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:37.119 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:37.137 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:37.137 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:37.141 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.141 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:37.154 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:37.154 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:37.218 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:37.220+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:37.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:16:37.267 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.267 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:37.299 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone12 from 0xcb61054 (overlap 0x9a591b6) to 0x8ae9cbd 2024-09-16T22:16:37.345 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:37.347+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:37.421 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:37.422+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:37.433 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.433 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:37.481 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T22:16:37.524 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:37.526+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:37.619 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.619 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.622 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:37.624+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:37.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:37.723 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:37.723 INFO:teuthology.orchestra.run.smithi031.stdout:1407 read 0xc4182e thru 0xc5001f (0xe7f2 bytes) 2024-09-16T22:16:37.723 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:37.725+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:37.723 INFO:teuthology.orchestra.run.smithi031.stdout:1408 write 0x7d1e8a5 thru 0x7d2b7c5 (0xcf21 bytes) 2024-09-16T22:16:37.790 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.790 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:37.823 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:37.825+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:37.855 INFO:teuthology.orchestra.run.smithi031.stdout:1409 read 0xc301ecb thru 0xc3065ea (0x4720 bytes) 2024-09-16T22:16:37.861 INFO:teuthology.orchestra.run.smithi031.stdout:1410 trunc from 0xcb61054 to 0x84ba7b3 2024-09-16T22:16:37.889 INFO:teuthology.orchestra.run.smithi031.stdout:1413 read 0x2ea8de5 thru 0x2eae9d3 (0x5bef bytes) 2024-09-16T22:16:37.890 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:37.897 INFO:teuthology.orchestra.run.smithi031.stdout:1415 trunc from 0x84ba7b3 to 0x1bdece6 2024-09-16T22:16:37.925 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:37.926+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:37.949 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:37.949 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:37.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:38.026 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:38.027+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:38.076 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.076 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.081 INFO:teuthology.orchestra.run.smithi031.stdout:1416 read 0x919077 thru 0x919dc1 (0xd4b bytes) 2024-09-16T22:16:38.081 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.081 INFO:teuthology.orchestra.run.smithi031.stdout:1418 read 0x3f1a11 thru 0x3f42dd (0x28cd bytes) 2024-09-16T22:16:38.084 INFO:teuthology.orchestra.run.smithi031.stdout:1419 trunc from 0x1bdece6 to 0x3b37689 2024-09-16T22:16:38.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:38.090 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.090 INFO:teuthology.orchestra.run.smithi031.stdout:1421 punch from 0x13ea8c9 to 0x13fa602, (0xfd39 bytes) 2024-09-16T22:16:38.097 INFO:teuthology.orchestra.run.smithi031.stdout:1422 write 0x58e9eac thru 0x58f75b2 (0xd707 bytes) 2024-09-16T22:16:38.106 INFO:teuthology.orchestra.run.smithi031.stdout:1423 trunc from 0x58f75b3 to 0x8463133 2024-09-16T22:16:38.111 INFO:teuthology.orchestra.run.smithi031.stdout:1424 writesame 0x21266a0 thru 0x21331ee data_size 0x3(0xcb4f bytes) 2024-09-16T22:16:38.121 INFO:teuthology.orchestra.run.smithi031.stdout:1425 read 0x83be3cc thru 0x83c3698 (0x52cd bytes) 2024-09-16T22:16:38.121 INFO:teuthology.orchestra.run.smithi031.stdout:1427 punch from 0x3179688 to 0x317df36, (0x48ae bytes) 2024-09-16T22:16:38.123 INFO:teuthology.orchestra.run.smithi031.stdout:1428 trunc from 0x8463133 to 0xa9a0f8e 2024-09-16T22:16:38.126 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:38.128+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:38.137 INFO:teuthology.orchestra.run.smithi031.stdout:1431 trunc from 0xa9a0f8e to 0x2aedfa7 2024-09-16T22:16:38.152 INFO:teuthology.orchestra.run.smithi031.stdout:1432 read 0xf2bf1e thru 0xf2f994 (0x3a77 bytes) 2024-09-16T22:16:38.171 INFO:teuthology.orchestra.run.smithi031.stdout:1433 writesame 0xd4dbffd thru 0xd4df50a data_size 0x1(0x350e bytes) 2024-09-16T22:16:38.180 INFO:teuthology.orchestra.run.smithi031.stdout:1434 trunc from 0xd4df50b to 0x500ba21 2024-09-16T22:16:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:1435 write 0xc6611a2 thru 0xc66124c (0xab bytes) 2024-09-16T22:16:38.203 INFO:teuthology.orchestra.run.smithi031.stdout:1437 punch from 0x7e87f56 to 0x7e8c0b0, (0x415a bytes) 2024-09-16T22:16:38.203 INFO:teuthology.orchestra.run.smithi031.stdout:1438 write 0x925bfa0 thru 0x9267e69 (0xbeca bytes) 2024-09-16T22:16:38.209 INFO:teuthology.orchestra.run.smithi031.stdout:1440 punch from 0x6f651a0 to 0x6f6768f, (0x24ef bytes) 2024-09-16T22:16:38.214 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.214 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.218 INFO:teuthology.orchestra.run.smithi031.stdout:1441 trunc from 0xc66124d to 0x4c0620a 2024-09-16T22:16:38.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:38.230 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.230 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.230 INFO:teuthology.orchestra.run.smithi031.stdout:1444 punch from 0x2e1ee7c to 0x2e2e540, (0xf6c4 bytes) 2024-09-16T22:16:38.230 INFO:teuthology.orchestra.run.smithi031.stdout:1445 read 0x19a09b0 thru 0x19a6455 (0x5aa6 bytes) 2024-09-16T22:16:38.230 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:38.232+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:38.231 INFO:teuthology.orchestra.run.smithi031.stdout:1447 trunc from 0x4c0620a to 0x4fc1e95 2024-09-16T22:16:38.236 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.236 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:16:38.246 INFO:teuthology.orchestra.run.smithi031.stdout:1451 trunc from 0x4fc1e95 to 0xe0efc38 2024-09-16T22:16:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:1452 read 0xd196ca5 thru 0xd199cf6 (0x3052 bytes) 2024-09-16T22:16:38.254 INFO:teuthology.orchestra.run.smithi031.stdout:1453 writesame 0x828ad88 thru 0x8295d4f data_size 0x7d(0xafc8 bytes) 2024-09-16T22:16:38.259 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.259 INFO:teuthology.orchestra.run.smithi031.stdout:1455 read 0x115821d thru 0x11594cb (0x12af bytes) 2024-09-16T22:16:38.260 INFO:teuthology.orchestra.run.smithi031.stdout:1456 read 0xb0a5378 thru 0xb0a6147 (0xdd0 bytes) 2024-09-16T22:16:38.260 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.274 INFO:teuthology.orchestra.run.smithi031.stdout:1458 trunc from 0xe0efc38 to 0x1a309ad 2024-09-16T22:16:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:1460 punch from 0x1324982 to 0x1330df5, (0xc473 bytes) 2024-09-16T22:16:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:1461 punch from 0x169e395 to 0x16a3807, (0x5472 bytes) 2024-09-16T22:16:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:1462 punch from 0x178accc to 0x178b789, (0xabd bytes) 2024-09-16T22:16:38.287 INFO:teuthology.orchestra.run.smithi031.stdout:1465 trunc from 0x1a309ad to 0x2271925 2024-09-16T22:16:38.306 INFO:teuthology.orchestra.run.smithi031.stdout:1468 write 0x8d358c1 thru 0x8d3691f (0x105f bytes) 2024-09-16T22:16:38.315 INFO:teuthology.orchestra.run.smithi031.stdout:1469 write 0x9dcd737 thru 0x9dd347a (0x5d44 bytes) 2024-09-16T22:16:38.317 INFO:teuthology.orchestra.run.smithi031.stdout:1470 read 0x3ef9603 thru 0x3efce84 (0x3882 bytes) 2024-09-16T22:16:38.318 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.325 INFO:teuthology.orchestra.run.smithi031.stdout:1473 writesame 0xb6eb555 thru 0xb6f971c data_size 0x64(0xe1c8 bytes) 2024-09-16T22:16:38.325 INFO:teuthology.orchestra.run.smithi031.stdout:1474 punch from 0xe9dba8 to 0xea4392, (0x67ea bytes) 2024-09-16T22:16:38.327 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:38.329+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:38.334 INFO:teuthology.orchestra.run.smithi031.stdout:1476 write 0xe740f4d thru 0xe75069e (0xf752 bytes) 2024-09-16T22:16:38.338 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.338 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.339 INFO:teuthology.orchestra.run.smithi031.stdout:1479 writesame 0x2f7b8f6 thru 0x2f7dfad data_size 0x3b(0x26b8 bytes) 2024-09-16T22:16:38.341 INFO:teuthology.orchestra.run.smithi031.stdout:1480 read 0x6b039b1 thru 0x6b08d66 (0x53b6 bytes) 2024-09-16T22:16:38.341 INFO:teuthology.orchestra.run.smithi031.stdout:1483 write 0x97f119 thru 0x985350 (0x6238 bytes) 2024-09-16T22:16:38.343 INFO:teuthology.orchestra.run.smithi031.stdout:1484 writesame 0x94b75ce thru 0x94bfa0d data_size 0x10(0x8440 bytes) 2024-09-16T22:16:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:1485 punch from 0xbaede2a to 0xbaf16e8, (0x38be bytes) 2024-09-16T22:16:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:1487 punch from 0x1ac4e42 to 0x1aca6c4, (0x5882 bytes) 2024-09-16T22:16:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:1488 write 0xde8436a thru 0xde8a2d9 (0x5f70 bytes) 2024-09-16T22:16:38.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:38.348 INFO:teuthology.orchestra.run.smithi031.stdout:1489 flatten 2024-09-16T22:16:38.353 INFO:teuthology.orchestra.run.smithi031.stdout:1490 read 0x4555bc5 thru 0x4564d5f (0xf19b bytes) 2024-09-16T22:16:38.353 INFO:teuthology.orchestra.run.smithi031.stdout:1491 write 0xc1f1d thru 0xc6073 (0x4157 bytes) 2024-09-16T22:16:38.355 INFO:teuthology.orchestra.run.smithi031.stdout:1492 read 0xcf859d0 thru 0xcf8ba00 (0x6031 bytes) 2024-09-16T22:16:38.356 INFO:teuthology.orchestra.run.smithi031.stdout:1493 write 0x291932e thru 0x292000d (0x6ce0 bytes) 2024-09-16T22:16:38.359 INFO:teuthology.orchestra.run.smithi031.stdout:1497 writesame 0xd43c87e thru 0xd445cec data_size 0x25(0x946f bytes) 2024-09-16T22:16:38.362 INFO:teuthology.orchestra.run.smithi031.stdout:1498 write 0x3060a22 thru 0x306ead9 (0xe0b8 bytes) 2024-09-16T22:16:38.363 INFO:teuthology.orchestra.run.smithi031.stdout:1500 read 0x3c0dfe4 thru 0x3c15fd2 (0x7fef bytes) 2024-09-16T22:16:38.364 INFO:teuthology.orchestra.run.smithi031.stdout:1503 read 0x44dcdc6 thru 0x44ec3a4 (0xf5df bytes) 2024-09-16T22:16:38.364 INFO:teuthology.orchestra.run.smithi031.stdout:1504 punch from 0xaa00911 to 0xaa0933d, (0x8a2c bytes) 2024-09-16T22:16:38.364 INFO:teuthology.orchestra.run.smithi031.stdout:1505 writesame 0x2a71462 thru 0x2a78488 data_size 0x1(0x7027 bytes) 2024-09-16T22:16:38.378 INFO:teuthology.orchestra.run.smithi031.stdout:1506 punch from 0x20aba9f to 0x20b934b, (0xd8ac bytes) 2024-09-16T22:16:38.378 INFO:teuthology.orchestra.run.smithi031.stdout:1507 writesame 0xeb8790 thru 0xebc3a5 data_size 0x1(0x3c16 bytes) 2024-09-16T22:16:38.387 INFO:teuthology.orchestra.run.smithi031.stdout:1508 write 0x1a501ab thru 0x1a5fa34 (0xf88a bytes) 2024-09-16T22:16:38.388 INFO:teuthology.orchestra.run.smithi031.stdout:1509 writesame 0xc6036f5 thru 0xc606033 data_size 0x1(0x293f bytes) 2024-09-16T22:16:38.393 INFO:teuthology.orchestra.run.smithi031.stdout:1510 read 0x522aa28 thru 0x523234e (0x7927 bytes) 2024-09-16T22:16:38.393 INFO:teuthology.orchestra.run.smithi031.stdout:1511 read 0x6e6b183 thru 0x6e732d6 (0x8154 bytes) 2024-09-16T22:16:38.393 INFO:teuthology.orchestra.run.smithi031.stdout:1512 read 0x8bbeec8 thru 0x8bcd390 (0xe4c9 bytes) 2024-09-16T22:16:38.394 INFO:teuthology.orchestra.run.smithi031.stdout:1513 trunc from 0xe75069f to 0xec65824 2024-09-16T22:16:38.399 INFO:teuthology.orchestra.run.smithi031.stdout:1514 read 0x5527325 thru 0x552ab76 (0x3852 bytes) 2024-09-16T22:16:38.399 INFO:teuthology.orchestra.run.smithi031.stdout:1516 read 0x2921ee1 thru 0x2929bd2 (0x7cf2 bytes) 2024-09-16T22:16:38.399 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.400 INFO:teuthology.orchestra.run.smithi031.stdout:1519 write 0xd2d681a thru 0xd2e6011 (0xf7f8 bytes) 2024-09-16T22:16:38.401 INFO:teuthology.orchestra.run.smithi031.stdout:1522 writesame 0x69dcd38 thru 0x69e5229 data_size 0x9a(0x84f2 bytes) 2024-09-16T22:16:38.404 INFO:teuthology.orchestra.run.smithi031.stdout:1523 read 0xe0711c1 thru 0xe0714d6 (0x316 bytes) 2024-09-16T22:16:38.404 INFO:teuthology.orchestra.run.smithi031.stdout:1524 writesame 0x5e4ace7 thru 0x5e4d45a data_size 0xa(0x2774 bytes) 2024-09-16T22:16:38.407 INFO:teuthology.orchestra.run.smithi031.stdout:1525 writesame 0x86f8887 thru 0x8705cbf data_size 0xb(0xd439 bytes) 2024-09-16T22:16:38.410 INFO:teuthology.orchestra.run.smithi031.stdout:1526 write 0x4410c60 thru 0x4411d3f (0x10e0 bytes) 2024-09-16T22:16:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:1527 write 0x7fa65e8 thru 0x7fa8170 (0x1b89 bytes) 2024-09-16T22:16:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:1529 read 0xe5e0ef1 thru 0xe5eeb5e (0xdc6e bytes) 2024-09-16T22:16:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:1530 write 0xe2ab624 thru 0xe2b74f6 (0xbed3 bytes) 2024-09-16T22:16:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:1532 writesame 0x2121adf thru 0x212775a data_size 0x6(0x5c7c bytes) 2024-09-16T22:16:38.418 INFO:teuthology.orchestra.run.smithi031.stdout:1534 write 0xcd7b880 thru 0xcd87ff4 (0xc775 bytes) 2024-09-16T22:16:38.428 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:38.430+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:38.430 INFO:teuthology.orchestra.run.smithi031.stdout:1536 trunc from 0xec65824 to 0x58de790 2024-09-16T22:16:38.447 INFO:teuthology.orchestra.run.smithi031.stdout:1537 trunc from 0x58de790 to 0x9bdb0aa 2024-09-16T22:16:38.453 INFO:teuthology.orchestra.run.smithi031.stdout:1538 write 0x4da8b04 thru 0x4db1cf3 (0x91f0 bytes) 2024-09-16T22:16:38.464 INFO:teuthology.orchestra.run.smithi031.stdout:1539 trunc from 0x9bdb0aa to 0x72d344 2024-09-16T22:16:38.480 INFO:teuthology.orchestra.run.smithi031.stdout:1540 read 0x1c4051 thru 0x1d0018 (0xbfc8 bytes) 2024-09-16T22:16:38.480 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.499 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.500 INFO:teuthology.orchestra.run.smithi031.stdout:1543 writesame 0xd425480 thru 0xd434616 data_size 0x1(0xf197 bytes) 2024-09-16T22:16:38.501 INFO:teuthology.orchestra.run.smithi031.stdout:1545 read 0x78f4f52 thru 0x78f5be6 (0xc95 bytes) 2024-09-16T22:16:38.501 INFO:teuthology.orchestra.run.smithi031.stdout:1546 punch from 0xb47f610 to 0xb4847ff, (0x51ef bytes) 2024-09-16T22:16:38.501 INFO:teuthology.orchestra.run.smithi031.stdout:1548 write 0x8e9494 thru 0x8ef4d4 (0x6041 bytes) 2024-09-16T22:16:38.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:1550 write 0xeb05583 thru 0xeb13652 (0xe0d0 bytes) 2024-09-16T22:16:38.512 INFO:teuthology.orchestra.run.smithi031.stdout:1551 write 0x6248638 thru 0x6251a8e (0x9457 bytes) 2024-09-16T22:16:38.517 INFO:teuthology.orchestra.run.smithi031.stdout:1552 read 0x4751e7 thru 0x47a4cd (0x52e7 bytes) 2024-09-16T22:16:38.521 INFO:teuthology.orchestra.run.smithi031.stdout:1553 trunc from 0xeb13653 to 0xbdd7afc 2024-09-16T22:16:38.533 INFO:teuthology.orchestra.run.smithi031.stdout:1554 trunc from 0xbdd7afc to 0xd3fe6b7 2024-09-16T22:16:38.539 INFO:teuthology.orchestra.run.smithi031.stdout:1556 read 0x636a7cf thru 0x636eb8f (0x43c1 bytes) 2024-09-16T22:16:38.539 INFO:teuthology.orchestra.run.smithi031.stdout:1558 write 0x260d8ea thru 0x26195d1 (0xbce8 bytes) 2024-09-16T22:16:38.543 INFO:teuthology.orchestra.run.smithi031.stdout:1559 read 0x668b5cd thru 0x668f59d (0x3fd1 bytes) 2024-09-16T22:16:38.543 INFO:teuthology.orchestra.run.smithi031.stdout:1562 read 0x6be70d3 thru 0x6bedf1d (0x6e4b bytes) 2024-09-16T22:16:38.544 INFO:teuthology.orchestra.run.smithi031.stdout:1564 punch from 0x880c3ed to 0x8814407, (0x801a bytes) 2024-09-16T22:16:38.544 INFO:teuthology.orchestra.run.smithi031.stdout:1565 read 0x1edad92 thru 0x1ee1df1 (0x7060 bytes) 2024-09-16T22:16:38.544 INFO:teuthology.orchestra.run.smithi031.stdout:1567 write 0x37cae19 thru 0x37d5531 (0xa719 bytes) 2024-09-16T22:16:38.552 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:38.554+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:38.554 INFO:teuthology.orchestra.run.smithi031.stdout:1569 trunc from 0xd3fe6b7 to 0x5a4b2db 2024-09-16T22:16:38.578 INFO:teuthology.orchestra.run.smithi031.stdout:1570 writesame 0xc02c308 thru 0xc02c3fe data_size 0x1(0xf7 bytes) 2024-09-16T22:16:38.578 INFO:teuthology.orchestra.run.smithi031.stdout:1572 write 0x6f6f57a thru 0x6f6f76e (0x1f5 bytes) 2024-09-16T22:16:38.582 INFO:teuthology.orchestra.run.smithi031.stdout:1573 punch from 0x3bbc127 to 0x3bbf91c, (0x37f5 bytes) 2024-09-16T22:16:38.582 INFO:teuthology.orchestra.run.smithi031.stdout:1574 read 0x4e24ba9 thru 0x4e27390 (0x27e8 bytes) 2024-09-16T22:16:38.582 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.582 INFO:teuthology.orchestra.run.smithi031.stdout:1576 writesame 0x22637e7 thru 0x2272a31 data_size 0x1(0xf24b bytes) 2024-09-16T22:16:38.602 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.608 INFO:teuthology.orchestra.run.smithi031.stdout:1578 write 0xc0b5fe2 thru 0xc0c47f4 (0xe813 bytes) 2024-09-16T22:16:38.615 INFO:teuthology.orchestra.run.smithi031.stdout:1579 write 0x74ac3a thru 0x74c7f3 (0x1bba bytes) 2024-09-16T22:16:38.618 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.618 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.624 INFO:teuthology.orchestra.run.smithi031.stdout:1581 trunc from 0xc0c47f5 to 0x4f9ef36 2024-09-16T22:16:38.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:38.630 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:38.631+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:38.636 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.642 INFO:teuthology.orchestra.run.smithi031.stdout:1583 writesame 0x58d5ac0 thru 0x58da0bf data_size 0xa(0x4600 bytes) 2024-09-16T22:16:38.642 INFO:teuthology.orchestra.run.smithi031.stdout:1584 write 0xa2603d thru 0xa2ebad (0x8b71 bytes) 2024-09-16T22:16:38.643 INFO:teuthology.orchestra.run.smithi031.stdout:1585 read 0x2f6f77f thru 0x2f7713c (0x79be bytes) 2024-09-16T22:16:38.644 INFO:teuthology.orchestra.run.smithi031.stdout:1587 punch from 0x19abbeb to 0x19aed5a, (0x316f bytes) 2024-09-16T22:16:38.644 INFO:teuthology.orchestra.run.smithi031.stdout:1588 write 0x3627992 thru 0x3628fc1 (0x1630 bytes) 2024-09-16T22:16:38.647 INFO:teuthology.orchestra.run.smithi031.stdout:1589 trunc from 0x58da0c0 to 0x748c550 2024-09-16T22:16:38.660 INFO:teuthology.orchestra.run.smithi031.stdout:1590 write 0x83b26c9 thru 0x83b345f (0xd97 bytes) 2024-09-16T22:16:38.663 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.671 INFO:teuthology.orchestra.run.smithi031.stdout:1592 trunc from 0x83b3460 to 0xe5ea11 2024-09-16T22:16:38.682 INFO:teuthology.orchestra.run.smithi031.stdout:1593 read 0x93ca46 thru 0x93cc8e (0x249 bytes) 2024-09-16T22:16:38.682 INFO:teuthology.orchestra.run.smithi031.stdout:1594 read 0xca9968 thru 0xcad87c (0x3f15 bytes) 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:1595 punch from 0x626e18 to 0x62a2ce, (0x34b6 bytes) 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:1597 read 0x627f54 thru 0x62b9c6 (0x3a73 bytes) 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:1598 punch from 0x5768ec to 0x57d29a, (0x69ae bytes) 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:1600 read 0x56bf63 thru 0x57113c (0x51da bytes) 2024-09-16T22:16:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:1601 read 0xb1a99c thru 0xb1db98 (0x31fd bytes) 2024-09-16T22:16:38.692 INFO:teuthology.orchestra.run.smithi031.stdout:1602 trunc from 0xe5ea11 to 0x8502f0b 2024-09-16T22:16:38.705 INFO:teuthology.orchestra.run.smithi031.stdout:1604 trunc from 0x8502f0b to 0xc53955 2024-09-16T22:16:38.712 INFO:teuthology.orchestra.run.smithi031.stdout:1605 read 0x7c3676 thru 0x7d2e3a (0xf7c5 bytes) 2024-09-16T22:16:38.728 INFO:teuthology.orchestra.run.smithi031.stdout:1606 writesame 0x8ddd721 thru 0x8dec7bf data_size 0x1(0xf09f bytes) 2024-09-16T22:16:38.728 INFO:teuthology.orchestra.run.smithi031.stdout:1607 read 0x2f26854 thru 0x2f2f59b (0x8d48 bytes) 2024-09-16T22:16:38.728 INFO:teuthology.orchestra.run.smithi031.stdout:1608 punch from 0x81e4379 to 0x81e754e, (0x31d5 bytes) 2024-09-16T22:16:38.728 INFO:teuthology.orchestra.run.smithi031.stdout:1612 clone 14 order 21 su 524288 sc 2 2024-09-16T22:16:38.731 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:38.733+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:38.773 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.773 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:38.832 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:38.834+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:38.924 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:38.924 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:38.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:38.933 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:38.935+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:39.034 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:39.036+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:39.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:39.134 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:39.136+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:39.150 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:39.150 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:39.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:39.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:39.235 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:39.236+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:39.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:39.303 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:39.306 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:39.306 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:39.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:39.321 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:39.335 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:39.337+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:39.339 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:39.356 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:39.375 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:39.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:39.401 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T22:16:39.412 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1373 3.b e28/23 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head v 28'7, mlcod=27'5)", 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.293602925999998, 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057964999999999996, 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129212+0000", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9700000000000003e-07 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129215+0000", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8830000000000002e-06 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129222+0000", 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8469999999999998e-06 2024-09-16T22:16:39.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129227+0000", 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4870000000000004e-06 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129279+0000", 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2219000000000003e-05 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129299+0000", 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9877999999999998e-05 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.134962+0000", 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056633079999999997 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135009+0000", 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6581e-05 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.830034678000001, 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:16:39.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:16:39.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:16:39.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1818 3.1 3:8772e73b:::rbd_data.11cd70c2eac3.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.702277680000002, 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062717169999999996, 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1818 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720537+0000", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720539+0000", 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.119e-06 2024-09-16T22:16:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720543+0000", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.464e-06 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720552+0000", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3090000000000001e-06 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720560+0000", 2024-09-16T22:16:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6909999999999997e-06 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720604+0000", 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3838999999999998e-05 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.720683+0000", 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8916000000000001e-05 2024-09-16T22:16:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.721544+0000", 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086141500000000003 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726702+0000", 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005157671 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726720+0000", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7717999999999999e-05 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726768+0000", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8035999999999999e-05 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.726809+0000", 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1539000000000003e-05 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1933 3.12 3:4b562b7a:::rbd_data.11cd70c2eac3.000000000000000d:head [truncate 1703936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.557724701000001, 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056569769999999997, 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1933 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865090+0000", 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865092+0000", 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6309999999999999e-06 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865097+0000", 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865116+0000", 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8766e-05 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865124+0000", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0099999999999995e-06 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865179+0000", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5169e-05 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865257+0000", 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8221999999999997e-05 2024-09-16T22:16:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.865377+0000", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119473 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.867211+0000", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00183428 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870680+0000", 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003468563 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870710+0000", 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0337000000000001e-05 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.870747+0000", 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7228000000000003e-05 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:9870 3.1 3:8781002f:::rbd_data.11cdc482f8dc.00000000000000b6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.617696492, 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057426480000000004, 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 9870 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805119+0000", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805127+0000", 2024-09-16T22:16:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6860000000000006e-06 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805137+0000", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7000000000000003e-06 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805144+0000", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805215+0000", 2024-09-16T22:16:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1155999999999994e-05 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805261+0000", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6279000000000002e-05 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805365+0000", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103929 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.805721+0000", 2024-09-16T22:16:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035559099999999999 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810764+0000", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050430689999999999 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810817+0000", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3029999999999999e-05 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.810861+0000", 2024-09-16T22:16:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4372999999999997e-05 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:12107 3.4 3:202b7b59:::rbd_data.11cdc482f8dc.0000000000000116:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.423024133, 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054483370000000001, 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12107 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999791+0000", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999792+0000", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9300000000000006e-07 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999796+0000", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4279999999999998e-06 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999807+0000", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0824e-05 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999813+0000", 2024-09-16T22:16:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9789999999999998e-06 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999863+0000", 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0127000000000003e-05 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:25.999976+0000", 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112828 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.005239+0000", 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052631580000000004 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13225 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.314328790999999, 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056928450000000002, 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13225 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108486+0000", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108487+0000", 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6899999999999996e-07 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108498+0000", 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0733999999999999e-05 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108509+0000", 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0553999999999999e-05 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108515+0000", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9480000000000002e-06 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108574+0000", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8906000000000001e-05 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108713+0000", 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013846 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114179+0000", 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054662740000000001 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13240 3.4 3:207ebefd:::rbd_data.11cdc482f8dc.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.308277683, 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057273300000000001, 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13240 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114537+0000", 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114538+0000", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114542+0000", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2710000000000003e-06 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114552+0000", 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6029999999999994e-06 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114564+0000", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1555e-05 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114611+0000", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7686999999999999e-05 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114691+0000", 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9938000000000006e-05 2024-09-16T22:16:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.114862+0000", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017119999999999999 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.119890+0000", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050279849999999996 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120170+0000", 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027961400000000002 2024-09-16T22:16:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120224+0000", 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3922000000000003e-05 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.120265+0000", 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0760000000000003e-05 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13339 3.12 3:4fd424ff:::rbd_data.11cdc482f8dc.0000000000000179:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.272160900999999, 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065394440000000002, 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13339 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150654+0000", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150656+0000", 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150665+0000", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8529999999999998e-06 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150685+0000", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0222999999999999e-05 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150697+0000", 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150774+0000", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6954999999999998e-05 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.150917+0000", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014322900000000001 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151076+0000", 2024-09-16T22:16:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015856799999999999 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.151448+0000", 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037228599999999999 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157138+0000", 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056900379999999997 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157169+0000", 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0910000000000001e-05 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.157194+0000", 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4542999999999999e-05 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13353 3.1 3:80996581:::rbd_data.11cdc482f8dc.0000000000000185:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.264640332999999, 2024-09-16T22:16:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054098480000000001, 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13353 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158175+0000", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158176+0000", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158180+0000", 2024-09-16T22:16:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158189+0000", 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9190000000000005e-06 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158196+0000", 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8340000000000001e-06 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158243+0000", 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7157999999999998e-05 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.158319+0000", 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5574000000000005e-05 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.163585+0000", 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052660169999999996 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13391 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.246162929, 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066380959999999996, 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13391 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176652+0000", 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176653+0000", 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.305e-06 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176658+0000", 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176670+0000", 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1982e-05 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176675+0000", 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1220000000000003e-06 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176719+0000", 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4036000000000003e-05 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176798+0000", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8981999999999997e-05 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183290+0000", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064919390000000004 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.23912713, 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:16:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:16:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:16:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13466 3.12 3:49b34a3f:::rbd_data.11cdc482f8dc.00000000000001bc:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.203399245, 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055848399999999998, 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13466 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219416+0000", 2024-09-16T22:16:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219419+0000", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6860000000000002e-06 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219426+0000", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0480000000000001e-06 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219431+0000", 2024-09-16T22:16:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219939+0000", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050803999999999997 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.220051+0000", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011102700000000001 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225001+0000", 2024-09-16T22:16:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049501299999999996 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.188612441, 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:16:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.175286351, 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13595 3.5 e32/23 3:a7f0f76d:::rbd_data.11cdc482f8dc.00000000000001fd:head v 32'67, mlcod=32'65)", 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.152319939, 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053849240000000001, 2024-09-16T22:16:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270495+0000", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270496+0000", 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270500+0000", 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0350000000000003e-06 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270508+0000", 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4970000000000001e-06 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270513+0000", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7720000000000002e-06 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270549+0000", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5667000000000003e-05 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.270567+0000", 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8507000000000001e-05 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275827+0000", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005260069 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275880+0000", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2831000000000003e-05 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8337878390000002, 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:16:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:16:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:16:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4557.0:16275 3.4 e35/23 ondisk, result = 0)", 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8172053259999998, 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084889960000000004, 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605610+0000", 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605611+0000", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3009999999999999e-06 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605616+0000", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605631+0000", 2024-09-16T22:16:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4827000000000001e-05 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.605640+0000", 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3840000000000003e-06 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614068+0000", 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084278679999999998 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614081+0000", 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2576e-05 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614099+0000", 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8108e-05 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:17626 3.12 3:49f8983c:::rbd_data.11cd82e047b7.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3080192~273] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9682329860000003, 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057507920000000002, 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 17626 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454582+0000", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.454584+0000", 2024-09-16T22:16:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7400000000000001e-06 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456196+0000", 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001612523 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456213+0000", 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6266999999999999e-05 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456221+0000", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.456e-06 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456302+0000", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1427999999999994e-05 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456399+0000", 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6055000000000001e-05 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.456573+0000", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017492300000000001 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.459141+0000", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002567747 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460269+0000", 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00112813 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460304+0000", 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4857000000000002e-05 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.460333+0000", 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8666e-05 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:23996 3.4 3:21e1bcc6:::rbd_data.11cd7b8f383d.000000000000017a:head [set-alloc-hint object_size 524288 write_size 524288,write 303104~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T22:16:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.750652+0000", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6721630140000001, 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054515320000000003, 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23996 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750652+0000", 2024-09-16T22:16:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750652+0000", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750653+0000", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750661+0000", 2024-09-16T22:16:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9990000000000008e-06 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750670+0000", 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8990000000000007e-06 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750676+0000", 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6710000000000002e-06 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750719+0000", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2619e-05 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750797+0000", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8487999999999998e-05 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.750882+0000", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5010000000000001e-05 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.753132+0000", 2024-09-16T22:16:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002249156 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.756013+0000", 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002881088 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.756065+0000", 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1814000000000002e-05 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.756104+0000", 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9048000000000003e-05 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.526 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:39.527 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:39.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:39.529 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:39.529+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:39.628 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:39.629+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:39.670 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:39.670 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:39.670 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:39.670 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:39.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:39.732 INFO:teuthology.orchestra.run.smithi031.stdout:1613 read 0xc312bd thru 0xc398df (0x8623 bytes) 2024-09-16T22:16:39.734 INFO:teuthology.orchestra.run.smithi031.stdout:1615 trunc from 0x8dec7c0 to 0x7e82d7b 2024-09-16T22:16:39.746 INFO:teuthology.orchestra.run.smithi031.stdout:1616 read 0x51d85e1 thru 0x51e02b8 (0x7cd8 bytes) 2024-09-16T22:16:39.746 INFO:teuthology.orchestra.run.smithi031.stdout:1617 clone 15 order 23 su 1048576 sc 3 2024-09-16T22:16:39.763 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:39.765+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:39.824 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:39.824 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:39.824 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:39.829 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:39.831+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:39.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:39.931 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:39.933+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:39.977 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1373 3.b 3:d7516fe2:::rbd_data.11cd70c2eac3.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.849335274000001, 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064652499999999996, 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1373 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128759+0000", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0299999999999998e-07 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128763+0000", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-06 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128773+0000", 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7010000000000008e-06 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128780+0000", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128825+0000", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5179000000000002e-05 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.128908+0000", 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3034999999999994e-05 2024-09-16T22:16:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.129097+0000", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018963500000000001 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.132248+0000", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031504860000000001 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135120+0000", 2024-09-16T22:16:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028716150000000001 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135178+0000", 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8621e-05 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.135224+0000", 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5593999999999997e-05 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1412 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'8, mlcod=28'6)", 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.798547556999999, 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049201369999999998, 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179546+0000", 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179547+0000", 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000004e-07 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179551+0000", 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3819999999999997e-06 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179562+0000", 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0567999999999999e-05 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179575+0000", 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3492000000000001e-05 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179614+0000", 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8284999999999997e-05 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.179627+0000", 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3618000000000001e-05 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184435+0000", 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048080299999999996 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.184466+0000", 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1232999999999997e-05 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:1894 3.0 e28/23 3:0772e4fa:::rbd_data.11cd70c2eac3.000000000000000e:head v 28'12, mlcod=28'10)", 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.182379832999999, 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055310519999999998, 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:39.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795714+0000", 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795715+0000", 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6700000000000003e-07 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795720+0000", 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795730+0000", 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9250000000000007e-06 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795736+0000", 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6500000000000001e-06 2024-09-16T22:16:39.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795782+0000", 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6669999999999999e-05 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.795801+0000", 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8202e-05 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801216+0000", 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054153120000000003 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.801245+0000", 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9122e-05 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13220 3.b 3:d58c95a5:::rbd_data.11cdc482f8dc.000000000000013f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.871667219000001, 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049084810000000001, 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13220 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106427+0000", 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1699999999999997e-07 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106432+0000", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106442+0000", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0137e-05 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106447+0000", 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8949999999999997e-06 2024-09-16T22:16:39.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106495+0000", 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8322000000000003e-05 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106569+0000", 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4054999999999995e-05 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.106749+0000", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180483 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.108197+0000", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014480020000000001 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111252+0000", 2024-09-16T22:16:39.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030545139999999999 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111303+0000", 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1406999999999998e-05 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.111335+0000", 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.167e-05 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:39.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13369 3.b 3:d34300fb:::rbd_data.11cdc482f8dc.0000000000000188:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.812779114, 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005364906, 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13369 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165315+0000", 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165316+0000", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.47e-07 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165320+0000", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2030000000000002e-06 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165330+0000", 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0448999999999999e-05 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165337+0000", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165380+0000", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2908000000000003e-05 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165446+0000", 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5754999999999996e-05 2024-09-16T22:16:40.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165546+0000", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100691 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170501+0000", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049542609999999997 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170589+0000", 2024-09-16T22:16:40.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8287000000000004e-05 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170632+0000", 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2985999999999998e-05 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170680+0000", 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7799000000000001e-05 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13370 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.812606200999999, 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005335607, 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13370 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165488+0000", 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.92e-07 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165491+0000", 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.056e-06 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165498+0000", 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.708e-06 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165502+0000", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1130000000000003e-06 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165731+0000", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000228936 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.165812+0000", 2024-09-16T22:16:40.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1353000000000005e-05 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.170823+0000", 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050107490000000001 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13378 3.0 e31/23 3:06bf120b:::rbd_data.11cdc482f8dc.000000000000018b:head v 31'30, mlcod=31'28)", 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.807023472000001, 2024-09-16T22:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005199207, 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171070+0000", 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171071+0000", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171075+0000", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171087+0000", 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1708e-05 2024-09-16T22:16:40.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171092+0000", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1460000000000004e-06 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171135+0000", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2388999999999997e-05 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171152+0000", 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708999999999999e-05 2024-09-16T22:16:40.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176231+0000", 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050785659999999996 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176270+0000", 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8766999999999997e-05 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13380 3.b 3:d5e8033c:::rbd_data.11cdc482f8dc.000000000000018c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.806897039000001, 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005174423, 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13380 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171197+0000", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171198+0000", 2024-09-16T22:16:40.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1500000000000003e-07 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171201+0000", 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7809999999999999e-06 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171220+0000", 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9000000000000001e-05 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171229+0000", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1149999999999994e-06 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171277+0000", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8062000000000002e-05 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171353+0000", 2024-09-16T22:16:40.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5986000000000001e-05 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171555+0000", 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202418 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.173540+0000", 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001984675 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176272+0000", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027319890000000002 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176317+0000", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4798e-05 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176371+0000", 2024-09-16T22:16:40.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4684e-05 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13381 3.6 e31/23 3:6539fbfd:::rbd_data.11cdc482f8dc.000000000000018e:head v 31'101, mlcod=31'99)", 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.806587166, 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047825259999999996, 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171507+0000", 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3499999999999996e-07 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171512+0000", 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0950000000000003e-06 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171528+0000", 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781999999999999e-05 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171535+0000", 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171580+0000", 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4663999999999998e-05 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.171591+0000", 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1026000000000001e-05 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176254+0000", 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046628090000000004 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176289+0000", 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5383999999999999e-05 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:13383 3.8 e31/23 3:129ab390:::rbd_data.11cdc482f8dc.0000000000000191:head v 31'100, mlcod=31'98)", 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.803947285, 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050429230000000004, 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174147+0000", 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174151+0000", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1489999999999996e-06 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174161+0000", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4380000000000001e-06 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174165+0000", 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2359999999999998e-06 2024-09-16T22:16:40.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174208+0000", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2738000000000003e-05 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.174225+0000", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7788e-05 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179148+0000", 2024-09-16T22:16:40.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049220849999999997 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.179189+0000", 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1909000000000001e-05 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13390 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.801487882, 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058595619999999996, 2024-09-16T22:16:40.030 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13390 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176606+0000", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176607+0000", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9800000000000003e-07 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176610+0000", 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6030000000000001e-06 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176620+0000", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999995e-06 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176625+0000", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-06 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176669+0000", 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3532000000000003e-05 2024-09-16T22:16:40.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.176744+0000", 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5045999999999999e-05 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182466+0000", 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005721673 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.034 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13397 3.b 3:d6344ef0:::rbd_data.11cdc482f8dc.0000000000000196:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.795338116, 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005279087, 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13397 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182756+0000", 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182757+0000", 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1299999999999998e-07 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182761+0000", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0709999999999996e-06 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182769+0000", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6589999999999996e-06 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182776+0000", 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:16:40.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182822+0000", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970999999999997e-05 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.182827+0000", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185510+0000", 2024-09-16T22:16:40.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026825009999999999 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185598+0000", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.831e-05 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.185814+0000", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021624099999999999 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186536+0000", 2024-09-16T22:16:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072142400000000002 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187969+0000", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014327859999999999 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.187998+0000", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9713000000000002e-05 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.188035+0000", 2024-09-16T22:16:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6516000000000001e-05 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.c epoch 32/23 rep_tid 368 entries 32'37 (0'0) error 3:3694cf9a:::rbd_data.11cdc482f8dc.00000000000001a1:head by client.4557.0:13410 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'35)", 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.783412781999999, 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043492569999999996, 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194681+0000", 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.48e-07 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194685+0000", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194699+0000", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3246e-05 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194710+0000", 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1351e-05 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.194764+0000", 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3566000000000002e-05 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.199030+0000", 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042666659999999997 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13438 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.771866058000001, 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053131510000000003, 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13438 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206228+0000", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206229+0000", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-06 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206235+0000", 2024-09-16T22:16:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.237e-06 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206248+0000", 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3192e-05 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206254+0000", 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8159999999999999e-06 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206294+0000", 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0533e-05 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.206378+0000", 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4405000000000002e-05 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211541+0000", 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051624430000000001 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13453 3.b 3:d218a8b0:::rbd_data.11cdc482f8dc.00000000000001af:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.766233445999999, 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050623450000000002, 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13453 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211860+0000", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211861+0000", 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0070000000000001e-06 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211867+0000", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1440000000000002e-06 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211880+0000", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3453e-05 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211889+0000", 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0089999999999996e-06 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.211939+0000", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9904999999999997e-05 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212026+0000", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7007000000000002e-05 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.212200+0000", 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017373700000000001 2024-09-16T22:16:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216837+0000", 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046371540000000001 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216867+0000", 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0380999999999999e-05 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216898+0000", 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0754999999999999e-05 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.216923+0000", 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4793000000000001e-05 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13469 3.b 3:d0ddb2a3:::rbd_data.11cdc482f8dc.00000000000001be:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.758387053, 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054339640000000003, 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13469 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219707+0000", 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219708+0000", 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.009e-06 2024-09-16T22:16:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219712+0000", 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219723+0000", 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1141999999999999e-05 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219730+0000", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1210000000000001e-06 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219790+0000", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9917999999999998e-05 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.219924+0000", 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133807 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.225141+0000", 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052165809999999996 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13602 3.b 3:d1d109b3:::rbd_data.11cdc482f8dc.00000000000001fe:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.704742712, 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050799549999999997, 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13602 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273351+0000", 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273352+0000", 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999995e-07 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273356+0000", 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.143e-06 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273365+0000", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9709999999999993e-06 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273371+0000", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273409+0000", 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7790000000000002e-05 2024-09-16T22:16:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273457+0000", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7754999999999999e-05 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.273562+0000", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104851 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.275934+0000", 2024-09-16T22:16:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023726329999999999 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278377+0000", 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024423790000000002 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278410+0000", 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3056999999999999e-05 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.278431+0000", 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1231000000000001e-05 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:23604 3.b 3:d4792668:::rbd_data.11cd3076d18c.0000000000000003:head [sparse-read 0~1441792] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5450333809999997, 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046028680000000004, 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23604 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433060+0000", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433061+0000", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8299999999999998e-07 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433063+0000", 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9410000000000001e-06 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433067+0000", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4750000000000004e-06 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.433072+0000", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1410000000000004e-06 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.435649+0000", 2024-09-16T22:16:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025763689999999998 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.435698+0000", 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9119000000000002e-05 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.437663+0000", 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019654400000000002 2024-09-16T22:16:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:23725 3.6 e39/23 3:6036e720:::rbd_data.11cd7b8f383d.0000000000000100:head v 39'330, mlcod=38'328)", 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.4319351979999997, 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047562580000000002, 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546159+0000", 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0599999999999999e-07 2024-09-16T22:16:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546165+0000", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546175+0000", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0081000000000001e-05 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546182+0000", 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9099999999999999e-06 2024-09-16T22:16:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546224+0000", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2358999999999999e-05 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546241+0000", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6614000000000001e-05 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550873+0000", 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046324189999999996 2024-09-16T22:16:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550915+0000", 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1708000000000001e-05 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:23726 3.0 e39/23 3:034d15c2:::rbd_data.11cd7b8f383d.0000000000000101:head v 39'50, mlcod=38'48)", 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.4317367790000004, 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045589699999999999, 2024-09-16T22:16:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546357+0000", 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546358+0000", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546363+0000", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7130000000000002e-06 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546373+0000", 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0019e-05 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546380+0000", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546430+0000", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9805000000000002e-05 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.546450+0000", 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550885+0000", 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044349719999999997 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:35.550916+0000", 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1211000000000003e-05 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:40.078 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:40.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:40.080 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:40.082+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:40.181 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:40.183+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:40.188 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:40.188 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:40.188 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:40.188 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:40.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:40.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:16:40.300 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:40.302+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:40.306 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone14 from 0x7e82d7b (overlap 0x7e82d7b) to 0x71dc288 2024-09-16T22:16:40.315 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:40.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:40.315 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:40.315 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:40.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:40.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:40.410 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:40.412+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:40.415 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T22:16:40.483 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:40.485+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:791 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.871816493000001, 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011713016999999999, 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 791 2024-09-16T22:16:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617605+0000", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.78e-07 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617610+0000", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617625+0000", 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5088e-05 2024-09-16T22:16:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617632+0000", 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1960000000000002e-06 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622247+0000", 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004614613 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622294+0000", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7210999999999999e-05 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622548+0000", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025380800000000001 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624807+0000", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022596249999999999 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629226+0000", 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044184940000000002 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629278+0000", 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2487000000000001e-05 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629317+0000", 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8454999999999997e-05 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:792 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.87164714, 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011640707, 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 792 2024-09-16T22:16:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617774+0000", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6499999999999999e-07 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617778+0000", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.656e-06 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617789+0000", 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0428000000000001e-05 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617795+0000", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.981e-06 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622907+0000", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051125449999999996 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622970+0000", 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2992999999999998e-05 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623170+0000", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000199252 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624855+0000", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001685256 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629324+0000", 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044690859999999997 2024-09-16T22:16:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629363+0000", 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9209999999999999e-05 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629414+0000", 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0634999999999999e-05 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:793 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.871121734999999, 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011223963999999999, 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 793 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618300+0000", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-06 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618306+0000", 2024-09-16T22:16:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618318+0000", 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1590000000000001e-05 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618326+0000", 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6200000000000005e-06 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623487+0000", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051610579999999996 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623539+0000", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1709999999999998e-05 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623742+0000", 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020280100000000001 2024-09-16T22:16:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624907+0000", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011651509999999999 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629432+0000", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045250020000000002 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629481+0000", 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9163999999999999e-05 2024-09-16T22:16:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629522+0000", 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1486000000000002e-05 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:794 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.871010194, 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011215777999999999, 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 794 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618411+0000", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0589999999999999e-06 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618417+0000", 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618429+0000", 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1627000000000001e-05 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618436+0000", 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4749999999999996e-06 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624022+0000", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055863629999999996 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624075+0000", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2540999999999997e-05 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624472+0000", 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039755500000000001 2024-09-16T22:16:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626817+0000", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023447849999999998 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629532+0000", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027150970000000001 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629584+0000", 2024-09-16T22:16:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1338999999999998e-05 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629626+0000", 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2168e-05 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:797 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.870811483000001, 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011182566999999999, 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 797 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618610+0000", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3e-07 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618614+0000", 2024-09-16T22:16:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4120000000000003e-06 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618624+0000", 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0051000000000001e-05 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618640+0000", 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6376999999999999e-05 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625267+0000", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066267929999999997 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625332+0000", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4689000000000004e-05 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625554+0000", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000222029 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629641+0000", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040872929999999997 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629697+0000", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5637000000000003e-05 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629751+0000", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3980000000000002e-05 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629791+0000", 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0575999999999999e-05 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:798 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.870579630000002, 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011102747, 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 798 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1900000000000002e-07 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618847+0000", 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1680000000000003e-06 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618858+0000", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1868000000000001e-05 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618865+0000", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625906+0000", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070405290000000002 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625998+0000", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2559999999999995e-05 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626397+0000", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039847599999999998 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629651+0000", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032540680000000002 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629842+0000", 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019086799999999999 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629899+0000", 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7666e-05 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629943+0000", 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4106000000000001e-05 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:800 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.870279125, 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011059087, 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 800 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619142+0000", 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4700000000000001e-07 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619153+0000", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0783e-05 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619164+0000", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1338999999999999e-05 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619171+0000", 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.579e-06 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626861+0000", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076901560000000001 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626956+0000", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4984999999999999e-05 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627511+0000", 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055479699999999999 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629995+0000", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024841479999999998 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630109+0000", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011454 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630160+0000", 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0371999999999998e-05 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630200+0000", 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0640999999999999e-05 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:804 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.870047502999999, 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010918458000000001, 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 804 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619374+0000", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619380+0000", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619393+0000", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2612999999999999e-05 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619400+0000", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627970+0000", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085695540000000001 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628019+0000", 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9518999999999998e-05 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628226+0000", 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020676000000000001 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630042+0000", 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001816493 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630211+0000", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016820600000000001 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630253+0000", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2281999999999999e-05 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630291+0000", 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8433999999999998e-05 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:805 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.869693064, 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010651252, 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 805 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619728+0000", 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0400000000000005e-07 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619733+0000", 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7829999999999998e-06 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619743+0000", 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7799999999999995e-06 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619749+0000", 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5699999999999998e-06 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628524+0000", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087750190000000002 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628601+0000", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6953000000000007e-05 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628948+0000", 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034638399999999998 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630098+0000", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011506140000000001 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630302+0000", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020387500000000001 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630343+0000", 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0839e-05 2024-09-16T22:16:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630378+0000", 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5531000000000003e-05 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1370 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.363386421000001, 2024-09-16T22:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092874129999999996, 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1370 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126038+0000", 2024-09-16T22:16:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126045+0000", 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126053+0000", 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5240000000000003e-06 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130267+0000", 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042138549999999999 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130362+0000", 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.522e-05 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130774+0000", 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041218299999999999 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.131499+0000", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072460999999999999 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135246+0000", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037469640000000002 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135284+0000", 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8572000000000003e-05 2024-09-16T22:16:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135321+0000", 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6847999999999997e-05 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13328 3.3 3:c2e7e3ca:::rbd_data.11cdc482f8dc.0000000000000177:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.344258702999999, 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091967899999999998, 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13328 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145162+0000", 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9599999999999999e-07 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145165+0000", 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.208e-06 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145173+0000", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6529999999999994e-06 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145179+0000", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145702+0000", 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052302200000000005 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.145820+0000", 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118544 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.154358+0000", 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008538159 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13334 3.15 3:a9fa41bc:::rbd_data.11cdc482f8dc.000000000000017b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.341972281, 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094895459999999997, 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13334 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147449+0000", 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147454+0000", 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6319999999999999e-06 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3657e-05 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147473+0000", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9800000000000003e-06 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147523+0000", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9397999999999997e-05 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147645+0000", 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012283699999999999 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.156938+0000", 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009292079 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13395 3.f 3:f350c6c8:::rbd_data.11cdc482f8dc.0000000000000194:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.308610653000001, 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014172446, 2024-09-16T22:16:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13395 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180811+0000", 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9500000000000001e-07 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180815+0000", 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1400000000000002e-06 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180825+0000", 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180830+0000", 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5350000000000004e-06 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180870+0000", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9391000000000003e-05 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180873+0000", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6380000000000002e-06 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186724+0000", 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058504869999999997 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186813+0000", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9585000000000004e-05 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.187027+0000", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021384099999999999 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.189456+0000", 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024287449999999999 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194872+0000", 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054157670000000002 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194949+0000", 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7781000000000006e-05 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194982+0000", 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2669999999999997e-05 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:29983 3.3 3:c564fef5:::rbd_data.11cd7b8f383d.000000000000000d:e [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0369349510000001, 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012142185, 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 29983 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452486+0000", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452490+0000", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452509+0000", 2024-09-16T22:16:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8383e-05 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452516+0000", 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2989999999999999e-06 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457283+0000", 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047664209999999999 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457441+0000", 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015812700000000001 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.464628+0000", 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071867859999999997 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30219 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0334928430000001, 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093140970000000003, 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30219 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455928+0000", 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455931+0000", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455939+0000", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5660000000000001e-06 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455945+0000", 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465066+0000", 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091213430000000005 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465148+0000", 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2033000000000006e-05 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465242+0000", 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3597999999999994e-05 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30228 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.033466505, 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094687899999999995, 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30228 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455956+0000", 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2630000000000002e-06 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455963+0000", 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8859999999999998e-06 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455969+0000", 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3240000000000002e-06 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465269+0000", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093002759999999997 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465352+0000", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3242e-05 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465423+0000", 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0369000000000003e-05 2024-09-16T22:16:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30245 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.033344542, 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095068010000000005, 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30245 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456077+0000", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000003e-07 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456080+0000", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.839e-06 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456090+0000", 2024-09-16T22:16:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3680000000000008e-06 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456096+0000", 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9719999999999997e-06 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465447+0000", 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093516729999999996 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465517+0000", 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9466000000000002e-05 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465583+0000", 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5685000000000006e-05 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30257 3.3 3:c564fef5:::rbd_data.11cd7b8f383d.000000000000000d:e [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.033102808, 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094097439999999994, 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30257 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456318+0000", 2024-09-16T22:16:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0079999999999999e-06 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456330+0000", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1168999999999999e-05 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456341+0000", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1337e-05 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456347+0000", 2024-09-16T22:16:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.387e-06 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465609+0000", 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092611410000000005 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465677+0000", 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465727+0000", 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0018000000000001e-05 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30330 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0319137860000001, 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092441629999999997, 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30330 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457507+0000", 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457509+0000", 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3259999999999998e-06 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457516+0000", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3049999999999998e-06 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457521+0000", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466606+0000", 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090854279999999996 2024-09-16T22:16:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466671+0000", 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5038999999999999e-05 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466751+0000", 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9537999999999996e-05 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30458 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.030591437, 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011604311000000001, 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30458 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458830+0000", 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458842+0000", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2172e-05 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458853+0000", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1045e-05 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458860+0000", 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-06 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470264+0000", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011404065999999999 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470344+0000", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9608000000000001e-05 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470433+0000", 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9035000000000004e-05 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:40.598 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:40.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:40.600 INFO:teuthology.orchestra.run.smithi031.stdout:1618 write 0x614aa54 thru 0x615790c (0xceb9 bytes) 2024-09-16T22:16:40.600 INFO:teuthology.orchestra.run.smithi031.stdout:1619 trunc from 0x7e82d7b to 0xa53b4cb 2024-09-16T22:16:40.600 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:40.600 INFO:teuthology.orchestra.run.smithi031.stdout:1621 read 0x4b39dc6 thru 0x4b4115e (0x7399 bytes) 2024-09-16T22:16:40.600 INFO:teuthology.orchestra.run.smithi031.stdout:1623 write 0xaccb23 thru 0xad1d75 (0x5253 bytes) 2024-09-16T22:16:40.600 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:40.600+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:40.636 INFO:teuthology.orchestra.run.smithi031.stdout:1624 read 0x5b42d36 thru 0x5b51467 (0xe732 bytes) 2024-09-16T22:16:40.636 INFO:teuthology.orchestra.run.smithi031.stdout:1625 read 0x7f6a7ff thru 0x7f6ba37 (0x1239 bytes) 2024-09-16T22:16:40.645 INFO:teuthology.orchestra.run.smithi031.stdout:1626 write 0xd566981 thru 0xd573ac2 (0xd142 bytes) 2024-09-16T22:16:40.648 INFO:teuthology.orchestra.run.smithi031.stdout:1627 read 0xbc28c18 thru 0xbc345c1 (0xb9aa bytes) 2024-09-16T22:16:40.648 INFO:teuthology.orchestra.run.smithi031.stdout:1628 read 0x3d38b82 thru 0x3d403dd (0x785c bytes) 2024-09-16T22:16:40.648 INFO:teuthology.orchestra.run.smithi031.stdout:1629 write 0x240a4d9 thru 0x240af03 (0xa2b bytes) 2024-09-16T22:16:40.653 INFO:teuthology.orchestra.run.smithi031.stdout:1630 write 0xb2c0bb2 thru 0xb2c6d06 (0x6155 bytes) 2024-09-16T22:16:40.656 INFO:teuthology.orchestra.run.smithi031.stdout:1631 writesame 0x40d101c thru 0x40d57a5 data_size 0x1(0x478a bytes) 2024-09-16T22:16:40.669 INFO:teuthology.orchestra.run.smithi031.stdout:1632 write 0x65f689d thru 0x66021c2 (0xb926 bytes) 2024-09-16T22:16:40.675 INFO:teuthology.orchestra.run.smithi031.stdout:1633 writesame 0x572b0ee thru 0x5730d4d data_size 0x10(0x5c60 bytes) 2024-09-16T22:16:40.680 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:40.680 INFO:teuthology.orchestra.run.smithi031.stdout:1638 write 0xb6b114c thru 0xb6bd206 (0xc0bb bytes) 2024-09-16T22:16:40.687 INFO:teuthology.orchestra.run.smithi031.stdout:1639 trunc from 0xd573ac3 to 0xa6f66a9 2024-09-16T22:16:40.697 INFO:teuthology.orchestra.run.smithi031.stdout:1640 writesame 0x5a1d519 thru 0x5a1f5b8 data_size 0x2(0x20a0 bytes) 2024-09-16T22:16:40.699 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:40.701+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:40.700 INFO:teuthology.orchestra.run.smithi031.stdout:1641 trunc from 0xa6f66a9 to 0xb50f18d 2024-09-16T22:16:40.705 INFO:teuthology.orchestra.run.smithi031.stdout:1642 read 0x1a8524c thru 0x1a8a91d (0x56d2 bytes) 2024-09-16T22:16:40.709 INFO:teuthology.orchestra.run.smithi031.stdout:1643 trunc from 0xb50f18d to 0x81c230a 2024-09-16T22:16:40.721 INFO:teuthology.orchestra.run.smithi031.stdout:1644 writesame 0x24083f7 thru 0x240a052 data_size 0x3c(0x1c5c bytes) 2024-09-16T22:16:40.722 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:40.722 INFO:teuthology.orchestra.run.smithi031.stdout:1647 read 0x7bc4ab2 thru 0x7bcf2ad (0xa7fc bytes) 2024-09-16T22:16:40.723 INFO:teuthology.orchestra.run.smithi031.stdout:1648 punch from 0x158761c to 0x158ec82, (0x7666 bytes) 2024-09-16T22:16:40.723 INFO:teuthology.orchestra.run.smithi031.stdout:1649 clone 16 order 21 su 1048576 sc 12 2024-09-16T22:16:40.752 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:40.752 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:40.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:40.801 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:40.803+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:40.901 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:40.903+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:40.907 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:40.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:41.002 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:41.004+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7042 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4591616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.493929056000001, 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052761957999999998, 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7042 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531308+0000", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531315+0000", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1470000000000004e-06 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531323+0000", 2024-09-16T22:16:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6979999999999998e-06 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531328+0000", 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2619999999999998e-06 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583754+0000", 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052425839000000002 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583827+0000", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2933000000000001e-05 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584069+0000", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024263 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7043 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4628480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.49389315, 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052969308, 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7043 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531344+0000", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531346+0000", 2024-09-16T22:16:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2520000000000002e-06 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531354+0000", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1329999999999999e-06 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531360+0000", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3380000000000004e-06 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584131+0000", 2024-09-16T22:16:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052771021000000001 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584233+0000", 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102227 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584313+0000", 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9849000000000001e-05 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7044 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4665344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.493869227999999, 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053119052999999999, 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7044 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531703+0000", 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033545599999999998 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531707+0000", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4959999999999998e-06 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531720+0000", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2830000000000001e-05 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531727+0000", 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1629999999999999e-06 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584342+0000", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052614840000000003 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584414+0000", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1876000000000001e-05 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584486+0000", 2024-09-16T22:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2391999999999994e-05 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7045 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4702208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.493489754000001, 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052901939000000002, 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7045 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531750+0000", 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531758+0000", 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7379999999999994e-06 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531775+0000", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.685e-05 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584514+0000", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052739923000000001 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584583+0000", 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8868999999999994e-05 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584649+0000", 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5377999999999996e-05 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7046 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4739072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.493451959, 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053033981000000001, 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533754+0000", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019694040000000001 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533759+0000", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533775+0000", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5441999999999999e-05 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533784+0000", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.884e-06 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584675+0000", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050891758000000002 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584750+0000", 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4411000000000003e-05 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584819+0000", 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8783999999999997e-05 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7055 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5070848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.491194906, 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052727892999999998, 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7055 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534044+0000", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.317e-06 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534051+0000", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.212e-06 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534056+0000", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586519+0000", 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052463981999999999 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586618+0000", 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586770+0000", 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000151368 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7056 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5107712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.491173251999999, 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052802007999999998, 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534064+0000", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534066+0000", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534072+0000", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534086+0000", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3360000000000001e-05 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586726+0000", 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052640575000000002 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586793+0000", 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6611000000000003e-05 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586865+0000", 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2321000000000001e-05 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7057 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5144576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.491140511999999, 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052927321999999999, 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-09-16T22:16:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536968+0000", 2024-09-16T22:16:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028722890000000001 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536973+0000", 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.865e-06 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536985+0000", 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2196e-05 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536993+0000", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7060000000000004e-06 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586893+0000", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049899678000000003 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586961+0000", 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8354000000000003e-05 2024-09-16T22:16:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.587023+0000", 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2234000000000001e-05 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7070 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5623808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.487930285999999, 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052526675000000002, 2024-09-16T22:16:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7070 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537307+0000", 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537309+0000", 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9429999999999999e-06 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537315+0000", 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537320+0000", 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589660+0000", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052340416000000001 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589747+0000", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6988000000000001e-05 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589833+0000", 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5739e-05 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7071 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5660672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.487910246, 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870691999999997, 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7071 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537327+0000", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537329+0000", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537337+0000", 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1769999999999998e-06 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537342+0000", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589872+0000", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052530514 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590113+0000", 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241179 2024-09-16T22:16:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590197+0000", 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3498000000000006e-05 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7072 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5697536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.487885156000001, 2024-09-16T22:16:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053032186000000002, 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7072 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537352+0000", 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537361+0000", 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1749999999999994e-06 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537370+0000", 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4560000000000006e-06 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537376+0000", 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6019999999999999e-06 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590237+0000", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05286101 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590309+0000", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1928999999999995e-05 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590384+0000", 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4606999999999999e-05 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7073 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5734400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.487851961000001, 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053168509000000003, 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537387+0000", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537394+0000", 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7089999999999996e-06 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537399+0000", 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9869999999999999e-06 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590419+0000", 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053019721999999998 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590481+0000", 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2317000000000007e-05 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590553+0000", 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2050000000000003e-05 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7074 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5771264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.487830769, 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053310116999999997, 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542944+0000", 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055381600000000003 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542949+0000", 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542961+0000", 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.203e-05 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542979+0000", 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7591000000000001e-05 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590580+0000", 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047601516000000003 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590647+0000", 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7044000000000004e-05 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590716+0000", 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8536999999999998e-05 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7150 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5042176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.481462413999999, 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052568581000000003, 2024-09-16T22:16:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7150 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543775+0000", 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543776+0000", 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.906e-06 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543783+0000", 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3990000000000001e-06 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543787+0000", 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596222+0000", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052435004 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596281+0000", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8764999999999997e-05 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596343+0000", 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1493000000000002e-05 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7152 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5079040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.481442393, 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052694566999999998, 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7152 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543795+0000", 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543803+0000", 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2560000000000002e-06 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543810+0000", 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8399999999999997e-06 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543815+0000", 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1420000000000001e-06 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596370+0000", 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052555367999999998 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596433+0000", 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2760000000000002e-05 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596489+0000", 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5819000000000002e-05 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7154 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5115904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.481414392, 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052806530999999997, 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7154 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543823+0000", 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.27e-07 2024-09-16T22:16:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543825+0000", 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543831+0000", 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543836+0000", 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596513+0000", 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052676730999999997 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596572+0000", 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9531999999999999e-05 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596629+0000", 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6536999999999998e-05 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7155 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5152768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.48139308, 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052931824000000002, 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7155 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543844+0000", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543846+0000", 2024-09-16T22:16:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0870000000000001e-06 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543853+0000", 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543858+0000", 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596654+0000", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052796211000000003 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596718+0000", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4046000000000003e-05 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596775+0000", 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7497999999999997e-05 2024-09-16T22:16:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7157 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5189632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.481372087, 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053064113000000003, 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7157 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548983+0000", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051182490000000001 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548987+0000", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548999+0000", 2024-09-16T22:16:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599e-05 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.549008+0000", 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999995e-06 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596801+0000", 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047793565000000003 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596860+0000", 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8761000000000002e-05 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596929+0000", 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8666999999999998e-05 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7229 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5148672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.471320425, 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052536309000000003, 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7229 2024-09-16T22:16:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553929+0000", 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2140000000000001e-05 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553936+0000", 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0550000000000002e-06 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553942+0000", 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606177+0000", 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052235072 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606379+0000", 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020166100000000001 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606452+0000", 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3753999999999999e-05 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7230 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.471286392, 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052682412999999997, 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7230 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557074+0000", 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031241379999999998 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557080+0000", 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2549999999999997e-06 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557091+0000", 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1121000000000001e-05 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557098+0000", 2024-09-16T22:16:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3799999999999996e-06 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606482+0000", 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049383972999999998 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606557+0000", 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5189000000000001e-05 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606633+0000", 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5357000000000004e-05 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.115 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:41.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:41.117 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:41.118+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:41.217 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:41.219+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:16:41.271 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:41.271 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:41.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:41.318 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:41.319+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:41.323 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone15 from 0x81c230a (overlap 0x7e82d7b) to 0x194d5e5 2024-09-16T22:16:41.410 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T22:16:41.419 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:41.421+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:41.440 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:41.441 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:41.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:41.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:41.542 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:41.544+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:41.567 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.567 INFO:teuthology.orchestra.run.smithi031.stdout:1652 writesame 0x611deca thru 0x612d484 data_size 0xd(0xf5bb bytes) 2024-09-16T22:16:41.583 INFO:teuthology.orchestra.run.smithi031.stdout:1653 writesame 0x8970cf6 thru 0x898054d data_size 0x24(0xf858 bytes) 2024-09-16T22:16:41.587 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.587 INFO:teuthology.orchestra.run.smithi031.stdout:1656 punch from 0x50e5315 to 0x50f4e80, (0xfb6b bytes) 2024-09-16T22:16:41.587 INFO:teuthology.orchestra.run.smithi031.stdout:1657 punch from 0x63dbbd to 0x642f7d, (0x53c0 bytes) 2024-09-16T22:16:41.595 INFO:teuthology.orchestra.run.smithi031.stdout:1658 writesame 0xa4c4762 thru 0xa4ca817 data_size 0x1(0x60b6 bytes) 2024-09-16T22:16:41.595 INFO:teuthology.orchestra.run.smithi031.stdout:1659 read 0x6fe9f8c thru 0x6ff7595 (0xd60a bytes) 2024-09-16T22:16:41.595 INFO:teuthology.orchestra.run.smithi031.stdout:1660 read 0x52f9369 thru 0x52fecf9 (0x5991 bytes) 2024-09-16T22:16:41.595 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.595 INFO:teuthology.orchestra.run.smithi031.stdout:1664 punch from 0x277d31e to 0x27835c2, (0x62a4 bytes) 2024-09-16T22:16:41.596 INFO:teuthology.orchestra.run.smithi031.stdout:1665 read 0xa2be1b5 thru 0xa2c0a02 (0x284e bytes) 2024-09-16T22:16:41.596 INFO:teuthology.orchestra.run.smithi031.stdout:1667 punch from 0x99a5584 to 0x99aa801, (0x527d bytes) 2024-09-16T22:16:41.596 INFO:teuthology.orchestra.run.smithi031.stdout:1668 punch from 0x8a077a5 to 0x8a07bba, (0x415 bytes) 2024-09-16T22:16:41.596 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.606 INFO:teuthology.orchestra.run.smithi031.stdout:1670 write 0xdbdf3fc thru 0xdbe8d93 (0x9998 bytes) 2024-09-16T22:16:41.610 INFO:teuthology.orchestra.run.smithi031.stdout:1671 punch from 0x59f7742 to 0x5a04f90, (0xd84e bytes) 2024-09-16T22:16:41.610 INFO:teuthology.orchestra.run.smithi031.stdout:1672 read 0x8f94ff9 thru 0x8f9cde3 (0x7deb bytes) 2024-09-16T22:16:41.611 INFO:teuthology.orchestra.run.smithi031.stdout:1673 writesame 0xaed121a thru 0xaed1719 data_size 0x14(0x500 bytes) 2024-09-16T22:16:41.614 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.614 INFO:teuthology.orchestra.run.smithi031.stdout:1675 punch from 0xcfda16 to 0xd0b833, (0xde1d bytes) 2024-09-16T22:16:41.620 INFO:teuthology.orchestra.run.smithi031.stdout:1677 writesame 0xdd259e5 thru 0xdd29563 data_size 0x1(0x3b7f bytes) 2024-09-16T22:16:41.620 INFO:teuthology.orchestra.run.smithi031.stdout:1678 punch from 0x68ac369 to 0x68aebbb, (0x2852 bytes) 2024-09-16T22:16:41.620 INFO:teuthology.orchestra.run.smithi031.stdout:1679 read 0x3922cb5 thru 0x392da0a (0xad56 bytes) 2024-09-16T22:16:41.620 INFO:teuthology.orchestra.run.smithi031.stdout:1680 read 0x1d4c7f thru 0x1dc610 (0x7992 bytes) 2024-09-16T22:16:41.621 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:41.623+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:41.621 INFO:teuthology.orchestra.run.smithi031.stdout:1682 trunc from 0xdd29564 to 0xe5ca690 2024-09-16T22:16:41.622 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17313 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2119430290000004, 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028206102, 2024-09-16T22:16:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17313 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411397+0000", 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411399+0000", 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411406+0000", 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7270000000000001e-06 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411412+0000", 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.331e-06 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439464+0000", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028051439000000001 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439551+0000", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7180000000000002e-05 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439603+0000", 2024-09-16T22:16:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1613999999999997e-05 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17314 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8437760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2119223810000008, 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028296733000000001, 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17314 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999998e-07 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411419+0000", 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3179999999999999e-06 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411423+0000", 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411426+0000", 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3079999999999999e-06 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439631+0000", 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028204916999999999 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439670+0000", 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8958000000000002e-05 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439714+0000", 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3451000000000001e-05 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17315 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8519680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2119092380000005, 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028395437999999999, 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17315 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411431+0000", 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411432+0000", 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0920000000000001e-06 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411435+0000", 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411438+0000", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1499999999999999e-06 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439741+0000", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028303423000000001 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439784+0000", 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2768000000000001e-05 2024-09-16T22:16:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439826+0000", 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17316 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8601600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118982230000004, 2024-09-16T22:16:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028532334999999999, 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17316 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411442+0000", 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7600000000000006e-07 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411445+0000", 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0340000000000001e-06 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411448+0000", 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6290000000000002e-06 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439858+0000", 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028409997999999999 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439898+0000", 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9672999999999998e-05 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439974+0000", 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5795000000000002e-05 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17317 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8683520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118876309999997, 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028635106, 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17317 2024-09-16T22:16:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411453+0000", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411456+0000", 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9359999999999999e-06 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411459+0000", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440000+0000", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028541183000000001 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440046+0000", 2024-09-16T22:16:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5833e-05 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440087+0000", 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0679999999999997e-05 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17318 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8765440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118770080000001, 2024-09-16T22:16:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028735130000000001, 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17318 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411463+0000", 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2399999999999999e-07 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411464+0000", 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0950000000000001e-06 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411467+0000", 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-06 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411469+0000", 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440114+0000", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028645204 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440157+0000", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2110999999999997e-05 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440198+0000", 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1032000000000003e-05 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17319 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118670900000001, 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028838682000000001, 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17319 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411473+0000", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2700000000000001e-07 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411474+0000", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411477+0000", 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9660000000000001e-06 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411479+0000", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4159999999999998e-06 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440225+0000", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028746163000000002 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440271+0000", 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5346999999999998e-05 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440311+0000", 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0500000000000002e-05 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17320 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8929280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.211857341, 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028955774, 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17320 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411484+0000", 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411486+0000", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.915e-06 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411489+0000", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440346+0000", 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028857256000000001 2024-09-16T22:16:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440397+0000", 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1091000000000003e-05 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440438+0000", 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0717999999999999e-05 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17321 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118472039999997, 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029072324, 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17321 2024-09-16T22:16:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411493+0000", 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4300000000000003e-07 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411494+0000", 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.561e-06 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411500+0000", 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6180000000000003e-06 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411506+0000", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5779999999999998e-06 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440465+0000", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028959748 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440504+0000", 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8214999999999999e-05 2024-09-16T22:16:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440565+0000", 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1061000000000004e-05 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17322 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9093120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118273530000003, 2024-09-16T22:16:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029196468, 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17322 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411513+0000", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411514+0000", 2024-09-16T22:16:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.894e-06 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411520+0000", 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411526+0000", 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5249999999999996e-06 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440597+0000", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029070833000000001 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440665+0000", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8402e-05 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440709+0000", 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3424000000000003e-05 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17323 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2118074350000008, 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029288971, 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17323 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411534+0000", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9209999999999999e-06 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411540+0000", 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0070000000000003e-06 2024-09-16T22:16:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411545+0000", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440738+0000", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029193447000000001 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440779+0000", 2024-09-16T22:16:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1047999999999997e-05 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440821+0000", 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1621e-05 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17324 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2117885259999994, 2024-09-16T22:16:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029399292, 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17324 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3099999999999998e-07 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411560+0000", 2024-09-16T22:16:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1330000000000005e-06 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411567+0000", 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411572+0000", 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440849+0000", 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029276555999999999 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440901+0000", 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2608000000000002e-05 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440950+0000", 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8971000000000003e-05 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17325 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9338880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2117592619999993, 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029483256999999999, 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17325 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417013+0000", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005432727 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417019+0000", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5599999999999999e-06 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417028+0000", 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-16T22:16:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417034+0000", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440980+0000", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02394655 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.441022+0000", 2024-09-16T22:16:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2290000000000003e-05 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.441063+0000", 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1032000000000003e-05 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17358 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2058999299999993, 2024-09-16T22:16:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028089057000000001, 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17358 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.41e-07 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417441+0000", 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0410000000000001e-06 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417445+0000", 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417448+0000", 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5330000000000001e-06 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445518+0000", 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028070674 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445529+0000", 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0319000000000001e-05 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17359 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2058887859999992, 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028100824, 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17359 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417452+0000", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417455+0000", 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.022e-06 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417457+0000", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445547+0000", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028089571000000001 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445551+0000", 2024-09-16T22:16:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5430000000000004e-06 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17360 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.205876366, 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028114694999999999, 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17360 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417465+0000", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.181e-06 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417468+0000", 2024-09-16T22:16:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1470000000000001e-06 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417470+0000", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445569+0000", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028098483 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445578+0000", 2024-09-16T22:16:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0950000000000002e-06 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17361 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2058657969999995, 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028118417, 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17361 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.28e-07 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417475+0000", 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417478+0000", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417480+0000", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445588+0000", 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028107783000000001 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445592+0000", 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0860000000000004e-06 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17362 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2058457110000003, 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134688000000001, 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17362 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7399999999999999e-07 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417496+0000", 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4980000000000001e-06 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417500+0000", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5399999999999997e-06 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417503+0000", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445620+0000", 2024-09-16T22:16:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116306000000001 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445628+0000", 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6479999999999992e-06 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17363 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2058320980000001, 2024-09-16T22:16:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028170649999999998, 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17363 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417508+0000", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417509+0000", 2024-09-16T22:16:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.119e-06 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417512+0000", 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0019999999999998e-06 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417514+0000", 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4040000000000002e-06 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445669+0000", 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028154708000000001 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445678+0000", 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1579999999999994e-06 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17364 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.205822221, 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028207459000000001, 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17364 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417518+0000", 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417519+0000", 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.04e-06 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417521+0000", 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8600000000000001e-06 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417524+0000", 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445714+0000", 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028190336999999999 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445725+0000", 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0451999999999999e-05 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:41.712 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:41.713 INFO:teuthology.orchestra.run.smithi031.stdout:1683 trunc from 0xe5ca690 to 0x95229fd 2024-09-16T22:16:41.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1684 read 0x106e365 thru 0x1074c21 (0x68bd bytes) 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1685 writesame 0x91da9f9 thru 0x91e56b6 data_size 0x1(0xacbe bytes) 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1686 read 0x744bf6b thru 0x744ed4e (0x2de4 bytes) 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1687 punch from 0x4e2803f to 0x4e2eb04, (0x6ac5 bytes) 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1689 write 0x4062e66 thru 0x4068b3b (0x5cd6 bytes) 2024-09-16T22:16:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:1691 trunc from 0x95229fd to 0xc45fdce 2024-09-16T22:16:41.716 INFO:teuthology.orchestra.run.smithi031.stdout:1693 writesame 0x6ae8871 thru 0x6af7559 data_size 0x1(0xece9 bytes) 2024-09-16T22:16:41.716 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:41.717 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:41.717 INFO:teuthology.orchestra.run.smithi031.stdout:1694 trunc from 0xc45fdce to 0xc784b3b 2024-09-16T22:16:41.720 INFO:teuthology.orchestra.run.smithi031.stdout:1695 punch from 0x331afb5 to 0x331ff91, (0x4fdc bytes) 2024-09-16T22:16:41.720 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.720 INFO:teuthology.orchestra.run.smithi031.stdout:1697 read 0x4d75ce0 thru 0x4d7891a (0x2c3b bytes) 2024-09-16T22:16:41.720 INFO:teuthology.orchestra.run.smithi031.stdout:1698 writesame 0x61bb7c5 thru 0x61c13bc data_size 0x3(0x5bf8 bytes) 2024-09-16T22:16:41.721 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:41.724+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:41.722 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:41.725 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.732 INFO:teuthology.orchestra.run.smithi031.stdout:1700 write 0xd254ee5 thru 0xd261f9c (0xd0b8 bytes) 2024-09-16T22:16:41.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:41.743 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:41.752 INFO:teuthology.orchestra.run.smithi031.stdout:1701 trunc from 0xd261f9d to 0x12f7b4 2024-09-16T22:16:41.766 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:41.766 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:41.785 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:41.785 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:41.791 INFO:teuthology.orchestra.run.smithi031.stdout:1702 write 0x8128ac4 thru 0x8130d34 (0x8271 bytes) 2024-09-16T22:16:41.794 INFO:teuthology.orchestra.run.smithi031.stdout:1703 write 0x3088406 thru 0x3098058 (0xfc53 bytes) 2024-09-16T22:16:41.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:41.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:41.806 INFO:teuthology.orchestra.run.smithi031.stdout:1704 trunc from 0x8130d35 to 0x204f888 2024-09-16T22:16:41.816 INFO:teuthology.orchestra.run.smithi031.stdout:1706 read 0x92f379 thru 0x93966c (0xa2f4 bytes) 2024-09-16T22:16:41.822 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:41.824+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:41.823 INFO:teuthology.orchestra.run.smithi031.stdout:1707 write 0x26a9c09 thru 0x26b6e10 (0xd208 bytes) 2024-09-16T22:16:41.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:41.826 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:41.842 INFO:teuthology.orchestra.run.smithi031.stdout:1708 write 0xaae455e thru 0xaae6877 (0x231a bytes) 2024-09-16T22:16:41.854 INFO:teuthology.orchestra.run.smithi031.stdout:1709 write 0xdf7533f thru 0xdf7b7f1 (0x64b3 bytes) 2024-09-16T22:16:41.857 INFO:teuthology.orchestra.run.smithi031.stdout:1710 write 0xdea8775 thru 0xdeacca6 (0x4532 bytes) 2024-09-16T22:16:41.858 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:41.858 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:41.860 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.860 INFO:teuthology.orchestra.run.smithi031.stdout:1712 writesame 0x6960963 thru 0x696dfd3 data_size 0x1d(0xd671 bytes) 2024-09-16T22:16:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:1713 read 0xb76a1b1 thru 0xb774c89 (0xaad9 bytes) 2024-09-16T22:16:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:1714 write 0x800cd9a thru 0x800f1ac (0x2413 bytes) 2024-09-16T22:16:41.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:1715 write 0x5fb7f1 thru 0x60b7a6 (0xffb6 bytes) 2024-09-16T22:16:41.888 INFO:teuthology.orchestra.run.smithi031.stdout:1716 trunc from 0xdf7b7f2 to 0x1ddd47 2024-09-16T22:16:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:1717 punch from 0x125ed2 to 0x12bb18, (0x5c46 bytes) 2024-09-16T22:16:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:1719 trunc from 0x1ddd47 to 0x20472e0 2024-09-16T22:16:41.923 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:41.925+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:41.923 INFO:teuthology.orchestra.run.smithi031.stdout:1721 writesame 0x309a005 thru 0x30a76e4 data_size 0x30(0xd6e0 bytes) 2024-09-16T22:16:41.941 INFO:teuthology.orchestra.run.smithi031.stdout:1722 write 0xede89a2 thru 0xedea27b (0x18da bytes) 2024-09-16T22:16:41.944 INFO:teuthology.orchestra.run.smithi031.stdout:1723 read 0x49fd246 thru 0x4a04b70 (0x792b bytes) 2024-09-16T22:16:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:1724 punch from 0xbbe5b9 to 0xbc9aeb, (0xb532 bytes) 2024-09-16T22:16:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:1726 write 0x6e1038a thru 0x6e1892a (0x85a1 bytes) 2024-09-16T22:16:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:1727 write 0xd49edec thru 0xd4a020c (0x1421 bytes) 2024-09-16T22:16:41.954 INFO:teuthology.orchestra.run.smithi031.stdout:1728 write 0x235fc72 thru 0x2369f01 (0xa290 bytes) 2024-09-16T22:16:41.957 INFO:teuthology.orchestra.run.smithi031.stdout:1731 read 0xdcf10ec thru 0xdcf6786 (0x569b bytes) 2024-09-16T22:16:41.957 INFO:teuthology.orchestra.run.smithi031.stdout:1732 write 0xd62d14d thru 0xd62e3d2 (0x1286 bytes) 2024-09-16T22:16:41.964 INFO:teuthology.orchestra.run.smithi031.stdout:1734 trunc from 0xedea27c to 0xb0318a9 2024-09-16T22:16:41.973 INFO:teuthology.orchestra.run.smithi031.stdout:1736 punch from 0x7ef0b5a to 0x7ef0f0f, (0x3b5 bytes) 2024-09-16T22:16:41.973 INFO:teuthology.orchestra.run.smithi031.stdout:1737 punch from 0x909e06a to 0x90a138f, (0x3325 bytes) 2024-09-16T22:16:41.973 INFO:teuthology.orchestra.run.smithi031.stdout:1738 read 0x6d65925 thru 0x6d71af8 (0xc1d4 bytes) 2024-09-16T22:16:41.973 INFO:teuthology.orchestra.run.smithi031.stdout:1739 read 0x756bd4f thru 0x756c827 (0xad9 bytes) 2024-09-16T22:16:41.974 INFO:teuthology.orchestra.run.smithi031.stdout:1740 write 0x7bb62ff thru 0x7bbcd0b (0x6a0d bytes) 2024-09-16T22:16:41.976 INFO:teuthology.orchestra.run.smithi031.stdout:1741 read 0x2ef3a78 thru 0x2ef749f (0x3a28 bytes) 2024-09-16T22:16:41.979 INFO:teuthology.orchestra.run.smithi031.stdout:1742 trunc from 0xb0318a9 to 0x830778b 2024-09-16T22:16:41.987 INFO:teuthology.orchestra.run.smithi031.stdout:1746 write 0x3e33963 thru 0x3e418bf (0xdf5d bytes) 2024-09-16T22:16:41.990 INFO:teuthology.orchestra.run.smithi031.stdout:1749 read 0x214a3b7 thru 0x2152517 (0x8161 bytes) 2024-09-16T22:16:41.990 INFO:teuthology.orchestra.run.smithi031.stdout:1750 read 0x1087232 thru 0x10896eb (0x24ba bytes) 2024-09-16T22:16:41.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:41.990 INFO:teuthology.orchestra.run.smithi031.stdout:1752 read 0x79f679d thru 0x79ff1f0 (0x8a54 bytes) 2024-09-16T22:16:41.998 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:41.998 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:41.999 INFO:teuthology.orchestra.run.smithi031.stdout:1754 trunc from 0x830778b to 0x37edc9 2024-09-16T22:16:42.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:42.010 INFO:teuthology.orchestra.run.smithi031.stdout:1755 read 0x289994 thru 0x29850a (0xeb77 bytes) 2024-09-16T22:16:42.015 INFO:teuthology.orchestra.run.smithi031.stdout:1756 trunc from 0x37edc9 to 0x4233803 2024-09-16T22:16:42.020 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:42.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T22:16:42.021 INFO:teuthology.orchestra.run.smithi031.stdout:1757 read 0x7e773 thru 0x861a8 (0x7a36 bytes) 2024-09-16T22:16:42.023 INFO:teuthology.orchestra.run.smithi031.stdout:1758 read 0x336a1e8 thru 0x336ed43 (0x4b5c bytes) 2024-09-16T22:16:42.023 INFO:teuthology.orchestra.run.smithi031.stdout:1759 write 0x14aac51 thru 0x14b380e (0x8bbe bytes) 2024-09-16T22:16:42.023 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:42.025+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:1760 read 0x2373cea thru 0x237d586 (0x989d bytes) 2024-09-16T22:16:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:1762 punch from 0x3cc7638 to 0x3cd47ce, (0xd196 bytes) 2024-09-16T22:16:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xee07ebe 2024-09-16T22:16:42.038 INFO:teuthology.orchestra.run.smithi031.stdout:1765 trunc from 0x4233803 to 0xee07ebe 2024-09-16T22:16:42.045 INFO:teuthology.orchestra.run.smithi031.stdout:1766 write 0x145f045 thru 0x1469c02 (0xabbe bytes) 2024-09-16T22:16:42.047 INFO:teuthology.orchestra.run.smithi031.stdout:1768 read 0x1309e40 thru 0x130d4b5 (0x3676 bytes) 2024-09-16T22:16:42.047 INFO:teuthology.orchestra.run.smithi031.stdout:1769 punch from 0xd2e527d to 0xd2e5846, (0x5c9 bytes) 2024-09-16T22:16:42.047 INFO:teuthology.orchestra.run.smithi031.stdout:1770 write 0xe3dce09 thru 0xe3e8ce6 (0xbede bytes) 2024-09-16T22:16:42.050 INFO:teuthology.orchestra.run.smithi031.stdout:1771 write 0xca960 thru 0xd3e86 (0x9527 bytes) 2024-09-16T22:16:42.053 INFO:teuthology.orchestra.run.smithi031.stdout:1772 punch from 0xe562f7c to 0xe5635f1, (0x675 bytes) 2024-09-16T22:16:42.053 INFO:teuthology.orchestra.run.smithi031.stdout:1773 read 0xe46418 thru 0xe49db1 (0x399a bytes) 2024-09-16T22:16:42.053 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.053 INFO:teuthology.orchestra.run.smithi031.stdout:1775 read 0x786e072 thru 0x786e8bc (0x84b bytes) 2024-09-16T22:16:42.053 INFO:teuthology.orchestra.run.smithi031.stdout:1776 writesame 0x6a3d2f9 thru 0x6a41fc0 data_size 0x27(0x4cc8 bytes) 2024-09-16T22:16:42.072 INFO:teuthology.orchestra.run.smithi031.stdout:1777 trunc from 0xee07ebe to 0x13b3bfc 2024-09-16T22:16:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:1778 read 0xb35e42 thru 0xb389a9 (0x2b68 bytes) 2024-09-16T22:16:42.101 INFO:teuthology.orchestra.run.smithi031.stdout:1779 writesame 0xa721577 thru 0xa7268df data_size 0x1(0x5369 bytes) 2024-09-16T22:16:42.101 INFO:teuthology.orchestra.run.smithi031.stdout:1781 read 0x53360ec thru 0x5339f84 (0x3e99 bytes) 2024-09-16T22:16:42.111 INFO:teuthology.orchestra.run.smithi031.stdout:1782 trunc from 0xa7268e0 to 0x197f9fb 2024-09-16T22:16:42.123 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:42.125+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:42.126 INFO:teuthology.orchestra.run.smithi031.stdout:1783 write 0x37bc938 thru 0x37bef88 (0x2651 bytes) 2024-09-16T22:16:42.129 INFO:teuthology.orchestra.run.smithi031.stdout:1784 punch from 0x3084843 to 0x30886fb, (0x3eb8 bytes) 2024-09-16T22:16:42.141 INFO:teuthology.orchestra.run.smithi031.stdout:1786 write 0x84ebd5d thru 0x84f7cec (0xbf90 bytes) 2024-09-16T22:16:42.145 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.145 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.150 INFO:teuthology.orchestra.run.smithi031.stdout:1787 trunc from 0x84f7ced to 0x34040f0 2024-09-16T22:16:42.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:42.176 INFO:teuthology.orchestra.run.smithi031.stdout:1789 trunc from 0x34040f0 to 0xabd88f5 2024-09-16T22:16:42.184 INFO:teuthology.orchestra.run.smithi031.stdout:1791 write 0x29730b4 thru 0x29754c2 (0x240f bytes) 2024-09-16T22:16:42.186 INFO:teuthology.orchestra.run.smithi031.stdout:1792 write 0x75d901f thru 0x75db8a3 (0x2885 bytes) 2024-09-16T22:16:42.190 INFO:teuthology.orchestra.run.smithi031.stdout:1793 read 0x5a8830a thru 0x5a975bc (0xf2b3 bytes) 2024-09-16T22:16:42.190 INFO:teuthology.orchestra.run.smithi031.stdout:1795 write 0x2a1370b thru 0x2a17abf (0x43b5 bytes) 2024-09-16T22:16:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:1797 writesame 0xe21f877 thru 0xe225df7 data_size 0x1(0x6581 bytes) 2024-09-16T22:16:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:1799 write 0x8c39dcb thru 0x8c44185 (0xa3bb bytes) 2024-09-16T22:16:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:1801 read 0x35e5ecd thru 0x35eb146 (0x527a bytes) 2024-09-16T22:16:42.215 INFO:teuthology.orchestra.run.smithi031.stdout:1803 writesame 0xe915115 thru 0xe9207cd data_size 0x1(0xb6b9 bytes) 2024-09-16T22:16:42.215 INFO:teuthology.orchestra.run.smithi031.stdout:1805 punch from 0x257ac71 to 0x25862df, (0xb66e bytes) 2024-09-16T22:16:42.215 INFO:teuthology.orchestra.run.smithi031.stdout:1806 punch from 0x6a5454e to 0x6a63cd1, (0xf783 bytes) 2024-09-16T22:16:42.215 INFO:teuthology.orchestra.run.smithi031.stdout:1808 write 0x6b82e10 thru 0x6b8f227 (0xc418 bytes) 2024-09-16T22:16:42.220 INFO:teuthology.orchestra.run.smithi031.stdout:1810 trunc from 0xe9207ce to 0xcb83c18 2024-09-16T22:16:42.224 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:42.225+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:42.226 INFO:teuthology.orchestra.run.smithi031.stdout:1811 writesame 0x164e723 thru 0x16531d9 data_size 0x1(0x4ab7 bytes) 2024-09-16T22:16:42.236 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.250 INFO:teuthology.orchestra.run.smithi031.stdout:1815 trunc from 0xcb83c18 to 0x98f85d 2024-09-16T22:16:42.266 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.266 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:42.286 INFO:teuthology.orchestra.run.smithi031.stdout:1816 writesame 0xce6d909 thru 0xce7011f data_size 0xb(0x2817 bytes) 2024-09-16T22:16:42.286 INFO:teuthology.orchestra.run.smithi031.stdout:1817 write 0x2ecc01a thru 0x2ed5ebb (0x9ea2 bytes) 2024-09-16T22:16:42.289 INFO:teuthology.orchestra.run.smithi031.stdout:1818 read 0x136ef67 thru 0x13701d5 (0x126f bytes) 2024-09-16T22:16:42.289 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.289 INFO:teuthology.orchestra.run.smithi031.stdout:1820 writesame 0xcda0d88 thru 0xcda2b67 data_size 0x10(0x1de0 bytes) 2024-09-16T22:16:42.292 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.292 INFO:teuthology.orchestra.run.smithi031.stdout:1822 writesame 0x45092de thru 0x451875f data_size 0x22(0xf482 bytes) 2024-09-16T22:16:42.305 INFO:teuthology.orchestra.run.smithi031.stdout:1823 trunc from 0xce70120 to 0x4e26bf1 2024-09-16T22:16:42.316 INFO:teuthology.orchestra.run.smithi031.stdout:1825 writesame 0x3272d27 thru 0x3274710 data_size 0x1f(0x19ea bytes) 2024-09-16T22:16:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:1827 trunc from 0x4e26bf1 to 0x19add71 2024-09-16T22:16:42.348 INFO:teuthology.orchestra.run.smithi031.stdout:1828 writesame 0x4b16b53 thru 0x4b25c0b data_size 0x1d(0xf0b9 bytes) 2024-09-16T22:16:42.348 INFO:teuthology.orchestra.run.smithi031.stdout:1830 read 0x10f31da thru 0x10f89d9 (0x5800 bytes) 2024-09-16T22:16:42.348 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.348 INFO:teuthology.orchestra.run.smithi031.stdout:1837 read 0x193a67d thru 0x193d613 (0x2f97 bytes) 2024-09-16T22:16:42.351 INFO:teuthology.orchestra.run.smithi031.stdout:1838 trunc from 0x4b25c0c to 0x8155cf0 2024-09-16T22:16:42.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.357 INFO:teuthology.orchestra.run.smithi031.stdout:1841 write 0x7de24d thru 0x7e949b (0xb24f bytes) 2024-09-16T22:16:42.358 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:42.360+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:42.367 INFO:teuthology.orchestra.run.smithi031.stdout:1842 trunc from 0x8155cf0 to 0x216e921 2024-09-16T22:16:42.373 INFO:teuthology.orchestra.run.smithi031.stdout:1843 read 0xf1e26a thru 0xf2c7e8 (0xe57f bytes) 2024-09-16T22:16:42.384 INFO:teuthology.orchestra.run.smithi031.stdout:1844 trunc from 0x216e921 to 0xc9e95c8 2024-09-16T22:16:42.397 INFO:teuthology.orchestra.run.smithi031.stdout:1845 write 0xeac6824 thru 0xead1018 (0xa7f5 bytes) 2024-09-16T22:16:42.401 INFO:teuthology.orchestra.run.smithi031.stdout:1846 read 0xd159b7c thru 0xd166f09 (0xd38e bytes) 2024-09-16T22:16:42.401 INFO:teuthology.orchestra.run.smithi031.stdout:1847 write 0x5325c0e thru 0x532bad6 (0x5ec9 bytes) 2024-09-16T22:16:42.405 INFO:teuthology.orchestra.run.smithi031.stdout:1848 write 0x17e9c22 thru 0x17f2c59 (0x9038 bytes) 2024-09-16T22:16:42.408 INFO:teuthology.orchestra.run.smithi031.stdout:1849 read 0xbe99667 thru 0xbe9cded (0x3787 bytes) 2024-09-16T22:16:42.418 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.418 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.424 INFO:teuthology.orchestra.run.smithi031.stdout:1850 trunc from 0xead1019 to 0x270d2f 2024-09-16T22:16:42.425 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:42.426+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:42.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:42.457 INFO:teuthology.orchestra.run.smithi031.stdout:1851 writesame 0xa65b8af thru 0xa666438 data_size 0x27(0xab8a bytes) 2024-09-16T22:16:42.461 INFO:teuthology.orchestra.run.smithi031.stdout:1852 trunc from 0xa666439 to 0x75b6591 2024-09-16T22:16:42.473 INFO:teuthology.orchestra.run.smithi031.stdout:1853 write 0x8ed4d19 thru 0x8ee33b2 (0xe69a bytes) 2024-09-16T22:16:42.476 INFO:teuthology.orchestra.run.smithi031.stdout:1854 writesame 0x29036b8 thru 0x290a604 data_size 0x1(0x6f4d bytes) 2024-09-16T22:16:42.491 INFO:teuthology.orchestra.run.smithi031.stdout:1855 read 0x44def1 thru 0x44e4a0 (0x5b0 bytes) 2024-09-16T22:16:42.492 INFO:teuthology.orchestra.run.smithi031.stdout:1856 writesame 0x70adf03 thru 0x70b511d data_size 0x5b(0x721b bytes) 2024-09-16T22:16:42.495 INFO:teuthology.orchestra.run.smithi031.stdout:1857 read 0x7cc212 thru 0x7d1f39 (0x5d28 bytes) 2024-09-16T22:16:42.495 INFO:teuthology.orchestra.run.smithi031.stdout:1858 write 0x716b10f thru 0x716f22b (0x411d bytes) 2024-09-16T22:16:42.503 INFO:teuthology.orchestra.run.smithi031.stdout:1859 write 0x96bff39 thru 0x96c4a21 (0x4ae9 bytes) 2024-09-16T22:16:42.506 INFO:teuthology.orchestra.run.smithi031.stdout:1860 read 0x10ff671 thru 0x110dea8 (0xe838 bytes) 2024-09-16T22:16:42.507 INFO:teuthology.orchestra.run.smithi031.stdout:1861 punch from 0x309e282 to 0x30a365e, (0x53dc bytes) 2024-09-16T22:16:42.507 INFO:teuthology.orchestra.run.smithi031.stdout:1862 punch from 0x6d46225 to 0x6d4a8c2, (0x469d bytes) 2024-09-16T22:16:42.507 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.507 INFO:teuthology.orchestra.run.smithi031.stdout:1864 read 0x42453a3 thru 0x425010a (0xad68 bytes) 2024-09-16T22:16:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:1865 writesame 0xac2cf11 thru 0xac2d746 data_size 0x1(0x836 bytes) 2024-09-16T22:16:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:1867 read 0xa3b2a5f thru 0xa3b9172 (0x6714 bytes) 2024-09-16T22:16:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:1870 punch from 0x549be91 to 0x54a3bc3, (0x7d32 bytes) 2024-09-16T22:16:42.523 INFO:teuthology.orchestra.run.smithi031.stdout:1871 write 0xd899938 thru 0xd8a5204 (0xb8cd bytes) 2024-09-16T22:16:42.526 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:42.528+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:42.527 INFO:teuthology.orchestra.run.smithi031.stdout:1872 trunc from 0xd8a5205 to 0xe56d8da 2024-09-16T22:16:42.534 INFO:teuthology.orchestra.run.smithi031.stdout:1873 read 0x7e08f3c thru 0x7e14518 (0xb5dd bytes) 2024-09-16T22:16:42.534 INFO:teuthology.orchestra.run.smithi031.stdout:1874 punch from 0x7dfa30d to 0x7dfd6f8, (0x33eb bytes) 2024-09-16T22:16:42.534 INFO:teuthology.orchestra.run.smithi031.stdout:1875 writesame 0xe276887 thru 0xe27e2ad data_size 0x1(0x7a27 bytes) 2024-09-16T22:16:42.551 INFO:teuthology.orchestra.run.smithi031.stdout:1877 punch from 0x978767f to 0x9793b99, (0xc51a bytes) 2024-09-16T22:16:42.551 INFO:teuthology.orchestra.run.smithi031.stdout:1881 write 0x8adfaf7 thru 0x8ae7cf2 (0x81fc bytes) 2024-09-16T22:16:42.554 INFO:teuthology.orchestra.run.smithi031.stdout:1882 read 0x99e785c thru 0x99f6284 (0xea29 bytes) 2024-09-16T22:16:42.554 INFO:teuthology.orchestra.run.smithi031.stdout:1883 writesame 0xbceb961 thru 0xbcf1ec7 data_size 0x11(0x6567 bytes) 2024-09-16T22:16:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:1884 write 0xe769425 thru 0xe76bb6c (0x2748 bytes) 2024-09-16T22:16:42.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.571 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.571 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.577 INFO:teuthology.orchestra.run.smithi031.stdout:1886 trunc from 0xe76bb6d to 0x7471f76 2024-09-16T22:16:42.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:1887 punch from 0x2974cf8 to 0x297dd89, (0x9091 bytes) 2024-09-16T22:16:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:1888 writesame 0x334c92e thru 0x335468f data_size 0xb(0x7d62 bytes) 2024-09-16T22:16:42.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.590 INFO:teuthology.orchestra.run.smithi031.stdout:1890 punch from 0x3e910b to 0x3f3346, (0xa23b bytes) 2024-09-16T22:16:42.590 INFO:teuthology.orchestra.run.smithi031.stdout:1891 write 0x6662935 thru 0x6670e7c (0xe548 bytes) 2024-09-16T22:16:42.593 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.593 INFO:teuthology.orchestra.run.smithi031.stdout:1899 trunc from 0x7471f76 to 0x7041720 2024-09-16T22:16:42.600 INFO:teuthology.orchestra.run.smithi031.stdout:1900 read 0x2d35190 thru 0x2d42aef (0xd960 bytes) 2024-09-16T22:16:42.600 INFO:teuthology.orchestra.run.smithi031.stdout:1901 read 0x15dc136 thru 0x15e2216 (0x60e1 bytes) 2024-09-16T22:16:42.600 INFO:teuthology.orchestra.run.smithi031.stdout:1902 writesame 0x522846 thru 0x524a9b data_size 0x6(0x2256 bytes) 2024-09-16T22:16:42.603 INFO:teuthology.orchestra.run.smithi031.stdout:1903 read 0x3576a04 thru 0x3583a30 (0xd02d bytes) 2024-09-16T22:16:42.611 INFO:teuthology.orchestra.run.smithi031.stdout:1904 writesame 0xa78ca5d thru 0xa78e104 data_size 0x32(0x16a8 bytes) 2024-09-16T22:16:42.616 INFO:teuthology.orchestra.run.smithi031.stdout:1905 trunc from 0xa78e105 to 0x6cc8351 2024-09-16T22:16:42.627 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:42.628+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:42.628 INFO:teuthology.orchestra.run.smithi031.stdout:1906 trunc from 0x6cc8351 to 0x2986e20 2024-09-16T22:16:42.650 INFO:teuthology.orchestra.run.smithi031.stdout:1908 writesame 0x91f1581 thru 0x91f290c data_size 0x9(0x138c bytes) 2024-09-16T22:16:42.658 INFO:teuthology.orchestra.run.smithi031.stdout:1910 write 0xc33d4a7 thru 0xc343ceb (0x6845 bytes) 2024-09-16T22:16:42.661 INFO:teuthology.orchestra.run.smithi031.stdout:1911 writesame 0xc1295e3 thru 0xc131cc2 data_size 0x68(0x86e0 bytes) 2024-09-16T22:16:42.666 INFO:teuthology.orchestra.run.smithi031.stdout:1913 write 0x6f8be44 thru 0x6f951d7 (0x9394 bytes) 2024-09-16T22:16:42.669 INFO:teuthology.orchestra.run.smithi031.stdout:1915 writesame 0x1e22432 thru 0x1e2f887 data_size 0x1(0xd456 bytes) 2024-09-16T22:16:42.704 INFO:teuthology.orchestra.run.smithi031.stdout:1916 write 0x45768db thru 0x4577a34 (0x115a bytes) 2024-09-16T22:16:42.715 INFO:teuthology.orchestra.run.smithi031.stdout:1917 write 0xd5923dd thru 0xd592b39 (0x75d bytes) 2024-09-16T22:16:42.718 INFO:teuthology.orchestra.run.smithi031.stdout:1919 read 0x1d822fe thru 0x1d890af (0x6db2 bytes) 2024-09-16T22:16:42.718 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.719 INFO:teuthology.orchestra.run.smithi031.stdout:1921 read 0xcc658ae thru 0xcc69e41 (0x4594 bytes) 2024-09-16T22:16:42.723 INFO:teuthology.orchestra.run.smithi031.stdout:1922 write 0xda2789d thru 0xda319fe (0xa162 bytes) 2024-09-16T22:16:42.727 INFO:teuthology.orchestra.run.smithi031.stdout:1923 write 0x33a4999 thru 0x33a4e55 (0x4bd bytes) 2024-09-16T22:16:42.727 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:42.729+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:42.729 INFO:teuthology.orchestra.run.smithi031.stdout:1924 read 0x6aaab2 thru 0x6b2d68 (0x82b7 bytes) 2024-09-16T22:16:42.729 INFO:teuthology.orchestra.run.smithi031.stdout:1926 punch from 0x1ac2f5e to 0x1acd510, (0xa5b2 bytes) 2024-09-16T22:16:42.729 INFO:teuthology.orchestra.run.smithi031.stdout:1929 punch from 0x5d894ac to 0x5d8d801, (0x4355 bytes) 2024-09-16T22:16:42.730 INFO:teuthology.orchestra.run.smithi031.stdout:1931 write 0x6671cb8 thru 0x6678e56 (0x719f bytes) 2024-09-16T22:16:42.732 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.732 INFO:teuthology.orchestra.run.smithi031.stdout:1933 read 0x9d48308 thru 0x9d57082 (0xed7b bytes) 2024-09-16T22:16:42.732 INFO:teuthology.orchestra.run.smithi031.stdout:1934 read 0x9cfc7bb thru 0x9d029fd (0x6243 bytes) 2024-09-16T22:16:42.732 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.732 INFO:teuthology.orchestra.run.smithi031.stdout:1937 write 0x81f093b thru 0x81f9b51 (0x9217 bytes) 2024-09-16T22:16:42.735 INFO:teuthology.orchestra.run.smithi031.stdout:1938 write 0x97bd459 thru 0x97c7797 (0xa33f bytes) 2024-09-16T22:16:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:42.751 INFO:teuthology.orchestra.run.smithi031.stdout:1939 trunc from 0xda319ff to 0x1731d66 2024-09-16T22:16:42.769 INFO:teuthology.orchestra.run.smithi031.stdout:1940 write 0x3351334 thru 0x335d24d (0xbf1a bytes) 2024-09-16T22:16:42.772 INFO:teuthology.orchestra.run.smithi031.stdout:1941 read 0x170465d thru 0x170e837 (0xa1db bytes) 2024-09-16T22:16:42.772 INFO:teuthology.orchestra.run.smithi031.stdout:1943 write 0x5bd1aa thru 0x5bf444 (0x229b bytes) 2024-09-16T22:16:42.788 INFO:teuthology.orchestra.run.smithi031.stdout:1944 write 0xb90dd2a thru 0xb911a67 (0x3d3e bytes) 2024-09-16T22:16:42.791 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:42.791 INFO:teuthology.orchestra.run.smithi031.stdout:1948 read 0x3b4b9e1 thru 0x3b4fb24 (0x4144 bytes) 2024-09-16T22:16:42.791 INFO:teuthology.orchestra.run.smithi031.stdout:1949 punch from 0xa3058b4 to 0xa30eec6, (0x9612 bytes) 2024-09-16T22:16:42.804 INFO:teuthology.orchestra.run.smithi031.stdout:1950 trunc from 0xb911a68 to 0x9eb4f 2024-09-16T22:16:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:1951 punch from 0x5b2a2 to 0x62a80, (0x77de bytes) 2024-09-16T22:16:42.827 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:42.829+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:42.832 INFO:teuthology.orchestra.run.smithi031.stdout:1952 write 0x7987ade thru 0x798f4c2 (0x79e5 bytes) 2024-09-16T22:16:42.835 INFO:teuthology.orchestra.run.smithi031.stdout:1953 write 0x3c69ed1 thru 0x3c756c4 (0xb7f4 bytes) 2024-09-16T22:16:42.838 INFO:teuthology.orchestra.run.smithi031.stdout:1955 write 0x5c0ff56 thru 0x5c153b6 (0x5461 bytes) 2024-09-16T22:16:42.852 INFO:teuthology.orchestra.run.smithi031.stdout:1957 write 0xcf83942 thru 0xcf8510b (0x17ca bytes) 2024-09-16T22:16:42.853 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:42.854 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:42.861 INFO:teuthology.orchestra.run.smithi031.stdout:1958 trunc from 0xcf8510c to 0x73bd663 2024-09-16T22:16:42.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:42.870 INFO:teuthology.orchestra.run.smithi031.stdout:1959 clone 17 order 18 su 32768 sc 3 2024-09-16T22:16:42.928 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:42.930+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:43.004 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.005 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:43.057 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:43.059+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:43.117 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.117 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:43.129 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:43.131+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:43.230 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:43.232+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:43.256 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T22:16:43.270 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.270 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:16:43.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T22:16:43.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:43.338 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone16 intact 2024-09-16T22:16:43.363 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:43.365+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:43.424 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.425 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.432 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:43.433+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:43.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:43.460 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T22:16:43.507 INFO:teuthology.orchestra.run.smithi031.stdout:1960 read 0x3cc04d2 thru 0x3ccb5e6 (0xb115 bytes) 2024-09-16T22:16:43.509 INFO:teuthology.orchestra.run.smithi031.stdout:1961 read 0x46c2e4e thru 0x46d0753 (0xd906 bytes) 2024-09-16T22:16:43.509 INFO:teuthology.orchestra.run.smithi031.stdout:1962 read 0x39fbae3 thru 0x39ffdcd (0x42eb bytes) 2024-09-16T22:16:43.509 INFO:teuthology.orchestra.run.smithi031.stdout:1964 read 0x1389e2a thru 0x138ad85 (0xf5c bytes) 2024-09-16T22:16:43.510 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:43.510 INFO:teuthology.orchestra.run.smithi031.stdout:1967 read 0x1a564e2 thru 0x1a5f156 (0x8c75 bytes) 2024-09-16T22:16:43.510 INFO:teuthology.orchestra.run.smithi031.stdout:1968 read 0xef3d thru 0xff8c (0x1050 bytes) 2024-09-16T22:16:43.528 INFO:teuthology.orchestra.run.smithi031.stdout:1969 write 0xe9224ca thru 0xe92baea (0x9621 bytes) 2024-09-16T22:16:43.532 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:43.534+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:43.535 INFO:teuthology.orchestra.run.smithi031.stdout:1970 writesame 0xe6ec72f thru 0xe6f01bf data_size 0x1d(0x3a91 bytes) 2024-09-16T22:16:43.542 INFO:teuthology.orchestra.run.smithi031.stdout:1972 punch from 0xdca7ca8 to 0xdcae273, (0x65cb bytes) 2024-09-16T22:16:43.542 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:43.542 INFO:teuthology.orchestra.run.smithi031.stdout:1975 writesame 0xb7c7c1 thru 0xb8bac8 data_size 0x65(0xf308 bytes) 2024-09-16T22:16:43.551 INFO:teuthology.orchestra.run.smithi031.stdout:1977 trunc from 0xe92baeb to 0xeaca67a 2024-09-16T22:16:43.560 INFO:teuthology.orchestra.run.smithi031.stdout:1978 trunc from 0xeaca67a to 0xb8a5508 2024-09-16T22:16:43.570 INFO:teuthology.orchestra.run.smithi031.stdout:1981 writesame 0x217c5e1 thru 0x21829ab data_size 0x1(0x63cb bytes) 2024-09-16T22:16:43.581 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.581 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.582 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:43.589 INFO:teuthology.orchestra.run.smithi031.stdout:1983 write 0xd1afd12 thru 0xd1b3842 (0x3b31 bytes) 2024-09-16T22:16:43.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:43.594 INFO:teuthology.orchestra.run.smithi031.stdout:1984 read 0x626db58 thru 0x6270cdc (0x3185 bytes) 2024-09-16T22:16:43.594 INFO:teuthology.orchestra.run.smithi031.stdout:1986 write 0x875ae2f thru 0x8768221 (0xd3f3 bytes) 2024-09-16T22:16:43.599 INFO:teuthology.orchestra.run.smithi031.stdout:1987 writesame 0xae14d71 thru 0xae21957 data_size 0x1(0xcbe7 bytes) 2024-09-16T22:16:43.620 INFO:teuthology.orchestra.run.smithi031.stdout:1989 read 0x142a6a5 thru 0x1438482 (0xddde bytes) 2024-09-16T22:16:43.620 INFO:teuthology.orchestra.run.smithi031.stdout:1991 read 0xcbcb8f3 thru 0xcbd6b7d (0xb28b bytes) 2024-09-16T22:16:43.620 INFO:teuthology.orchestra.run.smithi031.stdout:1992 read 0x38e9b86 thru 0x38f0335 (0x67b0 bytes) 2024-09-16T22:16:43.620 INFO:teuthology.orchestra.run.smithi031.stdout:1993 writesame 0x6d6a503 thru 0x6d6a9e9 data_size 0x1(0x4e7 bytes) 2024-09-16T22:16:43.621 INFO:teuthology.orchestra.run.smithi031.stdout:1995 clone 18 order 18 su 131072 sc 10 2024-09-16T22:16:43.634 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:43.636+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:43.742 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:43.744+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:43.835 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:43.836+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:43.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:43.844 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:43.844 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:43.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:43.936 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:43.937+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:43.969 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:43.987 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:43.987 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:43.997 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:43.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:43.997 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:43.997 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:44.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:44.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:44.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:44.021 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:44.021 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:44.037 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:44.038+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:44.038 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:44.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:44.055 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:44.072 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:44.088 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:44.089 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:44.116 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:44.116 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:44.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:44.138 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:44.139+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:44.239 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:44.240+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:16:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:791 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.661739403999999, 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011713016999999999, 2024-09-16T22:16:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 791 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617604+0000", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617605+0000", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.78e-07 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617610+0000", 2024-09-16T22:16:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617625+0000", 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5088e-05 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617632+0000", 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1960000000000002e-06 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622247+0000", 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004614613 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622294+0000", 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7210999999999999e-05 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622548+0000", 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025380800000000001 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624807+0000", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022596249999999999 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629226+0000", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044184940000000002 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629278+0000", 2024-09-16T22:16:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2487000000000001e-05 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629317+0000", 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8454999999999997e-05 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:792 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.661570050999998, 2024-09-16T22:16:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011640707, 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 792 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617773+0000", 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617774+0000", 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6499999999999999e-07 2024-09-16T22:16:44.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617778+0000", 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.656e-06 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617789+0000", 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0428000000000001e-05 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.617795+0000", 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.981e-06 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622907+0000", 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051125449999999996 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.622970+0000", 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2992999999999998e-05 2024-09-16T22:16:44.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623170+0000", 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000199252 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624855+0000", 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001685256 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629324+0000", 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044690859999999997 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629363+0000", 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9209999999999999e-05 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629414+0000", 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0634999999999999e-05 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:793 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.661044646000001, 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011223963999999999, 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 793 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618299+0000", 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618300+0000", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-06 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618306+0000", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618318+0000", 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1590000000000001e-05 2024-09-16T22:16:44.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618326+0000", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6200000000000005e-06 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623487+0000", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051610579999999996 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623539+0000", 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1709999999999998e-05 2024-09-16T22:16:44.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.623742+0000", 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020280100000000001 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624907+0000", 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011651509999999999 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629432+0000", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045250020000000002 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629481+0000", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9163999999999999e-05 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629522+0000", 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1486000000000002e-05 2024-09-16T22:16:44.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:794 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.660933105000002, 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011215777999999999, 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 794 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618410+0000", 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618411+0000", 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0589999999999999e-06 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618417+0000", 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618429+0000", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1627000000000001e-05 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618436+0000", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4749999999999996e-06 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624022+0000", 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055863629999999996 2024-09-16T22:16:44.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624075+0000", 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2540999999999997e-05 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.624472+0000", 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039755500000000001 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626817+0000", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023447849999999998 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629532+0000", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027150970000000001 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629584+0000", 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1338999999999998e-05 2024-09-16T22:16:44.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629626+0000", 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2168e-05 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:797 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.660734393999999, 2024-09-16T22:16:44.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011182566999999999, 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 797 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618609+0000", 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618610+0000", 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3e-07 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618614+0000", 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4120000000000003e-06 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618624+0000", 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0051000000000001e-05 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618640+0000", 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6376999999999999e-05 2024-09-16T22:16:44.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625267+0000", 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066267929999999997 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625332+0000", 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4689000000000004e-05 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625554+0000", 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000222029 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629641+0000", 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040872929999999997 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629697+0000", 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5637000000000003e-05 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629751+0000", 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3980000000000002e-05 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629791+0000", 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0575999999999999e-05 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:798 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.660502541, 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011102747, 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 798 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618841+0000", 2024-09-16T22:16:44.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1900000000000002e-07 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618847+0000", 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1680000000000003e-06 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618858+0000", 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1868000000000001e-05 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.618865+0000", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625906+0000", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070405290000000002 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.625998+0000", 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2559999999999995e-05 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626397+0000", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039847599999999998 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629651+0000", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032540680000000002 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629842+0000", 2024-09-16T22:16:44.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019086799999999999 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629899+0000", 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7666e-05 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629943+0000", 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4106000000000001e-05 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:800 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.660202036000001, 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011059087, 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 800 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619141+0000", 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619142+0000", 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4700000000000001e-07 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619153+0000", 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0783e-05 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619164+0000", 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1338999999999999e-05 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619171+0000", 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.579e-06 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626861+0000", 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076901560000000001 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.626956+0000", 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4984999999999999e-05 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627511+0000", 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055479699999999999 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.629995+0000", 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024841479999999998 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630109+0000", 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011454 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630160+0000", 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0371999999999998e-05 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630200+0000", 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0640999999999999e-05 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:804 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.659970414, 2024-09-16T22:16:44.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010918458000000001, 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 804 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619373+0000", 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619374+0000", 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619380+0000", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619393+0000", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2612999999999999e-05 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619400+0000", 2024-09-16T22:16:44.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.627970+0000", 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085695540000000001 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628019+0000", 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9518999999999998e-05 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628226+0000", 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020676000000000001 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630042+0000", 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001816493 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630211+0000", 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016820600000000001 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630253+0000", 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2281999999999999e-05 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630291+0000", 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8433999999999998e-05 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:805 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.659615975000001, 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010651252, 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 805 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619727+0000", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619728+0000", 2024-09-16T22:16:44.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0400000000000005e-07 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619733+0000", 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7829999999999998e-06 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619743+0000", 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7799999999999995e-06 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.619749+0000", 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5699999999999998e-06 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628524+0000", 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087750190000000002 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628601+0000", 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6953000000000007e-05 2024-09-16T22:16:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.628948+0000", 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034638399999999998 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630098+0000", 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011506140000000001 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630302+0000", 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020387500000000001 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630343+0000", 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0839e-05 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:21.630378+0000", 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5531000000000003e-05 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:1370 3.15 3:afb2caa7:::rbd_object_map.11cd70c2eac3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.153309331999999, 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092874129999999996, 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1370 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126034+0000", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126038+0000", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126045+0000", 2024-09-16T22:16:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1160000000000001e-06 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.126053+0000", 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5240000000000003e-06 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130267+0000", 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042138549999999999 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130362+0000", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.522e-05 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.130774+0000", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041218299999999999 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.131499+0000", 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072460999999999999 2024-09-16T22:16:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135246+0000", 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037469640000000002 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135284+0000", 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8572000000000003e-05 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:22.135321+0000", 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6847999999999997e-05 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13334 3.15 3:a9fa41bc:::rbd_data.11cdc482f8dc.000000000000017b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.131895192000002, 2024-09-16T22:16:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094895459999999997, 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13334 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147448+0000", 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147449+0000", 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-16T22:16:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147454+0000", 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6319999999999999e-06 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147467+0000", 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3657e-05 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147473+0000", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9800000000000003e-06 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147523+0000", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9397999999999997e-05 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.147645+0000", 2024-09-16T22:16:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012283699999999999 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.156938+0000", 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009292079 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:13395 3.f 3:f350c6c8:::rbd_data.11cdc482f8dc.0000000000000194:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.098533564, 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014172446, 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13395 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180810+0000", 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180811+0000", 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9500000000000001e-07 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180815+0000", 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1400000000000002e-06 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180825+0000", 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9709999999999999e-06 2024-09-16T22:16:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180830+0000", 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5350000000000004e-06 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180870+0000", 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9391000000000003e-05 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-16T22:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.180873+0000", 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6380000000000002e-06 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186724+0000", 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058504869999999997 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.186813+0000", 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9585000000000004e-05 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.187027+0000", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021384099999999999 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.189456+0000", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024287449999999999 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194872+0000", 2024-09-16T22:16:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054157670000000002 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194949+0000", 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7781000000000006e-05 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:26.194982+0000", 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2669999999999997e-05 2024-09-16T22:16:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:29983 3.3 3:c564fef5:::rbd_data.11cd7b8f383d.000000000000000d:e [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8268578619999998, 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012142185, 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 29983 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452485+0000", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452486+0000", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452490+0000", 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-16T22:16:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452509+0000", 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8383e-05 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.452516+0000", 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2989999999999999e-06 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457283+0000", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047664209999999999 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457441+0000", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015812700000000001 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.464628+0000", 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071867859999999997 2024-09-16T22:16:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30219 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.823415754, 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093140970000000003, 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30219 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455927+0000", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455928+0000", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455931+0000", 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-16T22:16:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455939+0000", 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5660000000000001e-06 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455945+0000", 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465066+0000", 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091213430000000005 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465148+0000", 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2033000000000006e-05 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465242+0000", 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3597999999999994e-05 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30228 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8233894160000004, 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094687899999999995, 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30228 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455954+0000", 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455956+0000", 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2630000000000002e-06 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455963+0000", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8859999999999998e-06 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.455969+0000", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3240000000000002e-06 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465269+0000", 2024-09-16T22:16:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093002759999999997 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465352+0000", 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3242e-05 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465423+0000", 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0369000000000003e-05 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30245 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8232674529999997, 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095068010000000005, 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30245 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456076+0000", 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456077+0000", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9800000000000003e-07 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456080+0000", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.839e-06 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456090+0000", 2024-09-16T22:16:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3680000000000008e-06 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456096+0000", 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9719999999999997e-06 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465447+0000", 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093516729999999996 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465517+0000", 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9466000000000002e-05 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465583+0000", 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5685000000000006e-05 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30257 3.3 3:c564fef5:::rbd_data.11cd7b8f383d.000000000000000d:e [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8230257190000003, 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094097439999999994, 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30257 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456317+0000", 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456318+0000", 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0079999999999999e-06 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456330+0000", 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1168999999999999e-05 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456341+0000", 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1337e-05 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.456347+0000", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.387e-06 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465609+0000", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092611410000000005 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465677+0000", 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.465727+0000", 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0018000000000001e-05 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30330 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8218366970000002, 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092441629999999997, 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30330 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457506+0000", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457507+0000", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457509+0000", 2024-09-16T22:16:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3259999999999998e-06 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457516+0000", 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3049999999999998e-06 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.457521+0000", 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466606+0000", 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090854279999999996 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466671+0000", 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5038999999999999e-05 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.466751+0000", 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9537999999999996e-05 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:30458 3.3 3:c49a28ba:::rbd_data.11cd7b8f383d.000000000000005c:e [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8205143479999997, 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011604311000000001, 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 30458 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458829+0000", 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458830+0000", 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458842+0000", 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2172e-05 2024-09-16T22:16:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458853+0000", 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1045e-05 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.458860+0000", 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-06 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470264+0000", 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011404065999999999 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470344+0000", 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9608000000000001e-05 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.470433+0000", 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9035000000000004e-05 2024-09-16T22:16:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:32944 3.c 3:3301c07a:::rbd_data.11cdcaf424fa.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:40.662269+0000", 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.617073977, 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092034209999999998, 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 32944 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662269+0000", 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662269+0000", 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662272+0000", 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3499999999999999e-06 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662283+0000", 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0981e-05 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662304+0000", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1233000000000002e-05 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662313+0000", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3049999999999998e-06 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662388+0000", 2024-09-16T22:16:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4602000000000002e-05 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.662501+0000", 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113146 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.664235+0000", 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017342939999999999 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.671374+0000", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071393489999999997 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.671386+0000", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2016e-05 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.671421+0000", 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4533000000000003e-05 2024-09-16T22:16:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:40.671473+0000", 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1612e-05 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.395 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:44.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T22:16:44.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:44.398 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone17 from 0xd1b3843 (overlap 0x73bd663) to 0x1725e13 2024-09-16T22:16:44.399 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:44.400+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:44.501 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:44.502+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:44.601 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:44.603+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:44.610 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:44.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:44.661 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T22:16:44.704 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:44.706+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:44.804 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:44.804+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:44.804 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:44.804 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:44.804 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:44.805 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:44.805 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:44.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:44.817 INFO:teuthology.orchestra.run.smithi031.stdout:1996 read 0x48700 thru 0x532a3 (0xaba4 bytes) 2024-09-16T22:16:44.820 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:44.820 INFO:teuthology.orchestra.run.smithi031.stdout:1998 write 0x12007e3 thru 0x120c384 (0xbba2 bytes) 2024-09-16T22:16:44.826 INFO:teuthology.orchestra.run.smithi031.stdout:1999 write 0x5aa8546 thru 0x5aabb4b (0x3606 bytes) 2024-09-16T22:16:44.835 INFO:teuthology.orchestra.run.smithi031.stdout:2001 write 0xdbf9665 thru 0xdc060d7 (0xca73 bytes) 2024-09-16T22:16:44.839 INFO:teuthology.orchestra.run.smithi031.stdout:2002 write 0xca3c726 thru 0xca3f230 (0x2b0b bytes) 2024-09-16T22:16:44.842 INFO:teuthology.orchestra.run.smithi031.stdout:2004 flatten 2024-09-16T22:16:44.902 INFO:teuthology.orchestra.run.smithi031.stdout:2005 trunc from 0xdc060d8 to 0x4aae309 2024-09-16T22:16:44.904 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:44.905+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:2006 read 0x4371810 thru 0x437a283 (0x8a74 bytes) 2024-09-16T22:16:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:2007 write 0x2c2ca9d thru 0x2c3c942 (0xfea6 bytes) 2024-09-16T22:16:44.925 INFO:teuthology.orchestra.run.smithi031.stdout:2009 write 0x6dd154f thru 0x6ddc0e5 (0xab97 bytes) 2024-09-16T22:16:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:2010 punch from 0x22a99d6 to 0x22b8185, (0xe7af bytes) 2024-09-16T22:16:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:2012 read 0x32ead4d thru 0x32f0bd7 (0x5e8b bytes) 2024-09-16T22:16:44.928 INFO:teuthology.orchestra.run.smithi031.stdout:2013 writesame 0x2fa7b0f thru 0x2fadb02 data_size 0x17(0x5ff4 bytes) 2024-09-16T22:16:44.942 INFO:teuthology.orchestra.run.smithi031.stdout:2015 write 0xcdfc4b3 thru 0xce06bd6 (0xa724 bytes) 2024-09-16T22:16:44.949 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:44.950 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:44.950 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:44.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:44.950 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:44.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7042 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4591616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.420097123000001, 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052761957999999998, 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7042 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531308+0000", 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531315+0000", 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1470000000000004e-06 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531323+0000", 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6979999999999998e-06 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531328+0000", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2619999999999998e-06 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583754+0000", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052425839000000002 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583827+0000", 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2933000000000001e-05 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584069+0000", 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024263 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7043 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4628480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.420061217000001, 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052969308, 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7043 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531344+0000", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531346+0000", 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2520000000000002e-06 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531354+0000", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1329999999999999e-06 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531360+0000", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3380000000000004e-06 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584131+0000", 2024-09-16T22:16:44.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052771021000000001 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584233+0000", 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102227 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584313+0000", 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9849000000000001e-05 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7044 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4665344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.420037295, 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053119052999999999, 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7044 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531703+0000", 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033545599999999998 2024-09-16T22:16:44.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531707+0000", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4959999999999998e-06 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531720+0000", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2830000000000001e-05 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531727+0000", 2024-09-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1629999999999999e-06 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584342+0000", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052614840000000003 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584414+0000", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1876000000000001e-05 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584486+0000", 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2391999999999994e-05 2024-09-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7045 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4702208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.419657821000001, 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052901939000000002, 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7045 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531750+0000", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531758+0000", 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7379999999999994e-06 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531775+0000", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.685e-05 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584514+0000", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052739923000000001 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584583+0000", 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8868999999999994e-05 2024-09-16T22:16:44.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584649+0000", 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5377999999999996e-05 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7046 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4739072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:44.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.419620026, 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053033981000000001, 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533754+0000", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019694040000000001 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533759+0000", 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533775+0000", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5441999999999999e-05 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533784+0000", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.884e-06 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584675+0000", 2024-09-16T22:16:44.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050891758000000002 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584750+0000", 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4411000000000003e-05 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584819+0000", 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8783999999999997e-05 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7055 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5070848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.417362972999999, 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052727892999999998, 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7055 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534044+0000", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.317e-06 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534051+0000", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.212e-06 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534056+0000", 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-16T22:16:44.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586519+0000", 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052463981999999999 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586618+0000", 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586770+0000", 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000151368 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7056 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5107712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.417341318999998, 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052802007999999998, 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534064+0000", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534066+0000", 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534072+0000", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534086+0000", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3360000000000001e-05 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586726+0000", 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052640575000000002 2024-09-16T22:16:44.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586793+0000", 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6611000000000003e-05 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586865+0000", 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2321000000000001e-05 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7057 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5144576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.417308579, 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052927321999999999, 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536968+0000", 2024-09-16T22:16:44.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028722890000000001 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536973+0000", 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.865e-06 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536985+0000", 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2196e-05 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536993+0000", 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7060000000000004e-06 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586893+0000", 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049899678000000003 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586961+0000", 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8354000000000003e-05 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.587023+0000", 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2234000000000001e-05 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7070 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5623808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.414098353, 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052526675000000002, 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7070 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537307+0000", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537309+0000", 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9429999999999999e-06 2024-09-16T22:16:44.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537315+0000", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537320+0000", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589660+0000", 2024-09-16T22:16:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052340416000000001 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589747+0000", 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6988000000000001e-05 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589833+0000", 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5739e-05 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7071 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5660672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.414078313000001, 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870691999999997, 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7071 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537327+0000", 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537329+0000", 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537337+0000", 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1769999999999998e-06 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537342+0000", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589872+0000", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052530514 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590113+0000", 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241179 2024-09-16T22:16:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590197+0000", 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3498000000000006e-05 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7072 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5697536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.414053223, 2024-09-16T22:16:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053032186000000002, 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7072 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537352+0000", 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537361+0000", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1749999999999994e-06 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537370+0000", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4560000000000006e-06 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537376+0000", 2024-09-16T22:16:44.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6019999999999999e-06 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590237+0000", 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05286101 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590309+0000", 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1928999999999995e-05 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:44.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590384+0000", 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4606999999999999e-05 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7073 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5734400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.414020027999999, 2024-09-16T22:16:45.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053168509000000003, 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537387+0000", 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537394+0000", 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7089999999999996e-06 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537399+0000", 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9869999999999999e-06 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590419+0000", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053019721999999998 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590481+0000", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2317000000000007e-05 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590553+0000", 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2050000000000003e-05 2024-09-16T22:16:45.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7074 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5771264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.413998836000001, 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053310116999999997, 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542944+0000", 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055381600000000003 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542949+0000", 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542961+0000", 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.203e-05 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542979+0000", 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7591000000000001e-05 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590580+0000", 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047601516000000003 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590647+0000", 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7044000000000004e-05 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590716+0000", 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8536999999999998e-05 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7150 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5042176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.407630480999998, 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052568581000000003, 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7150 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543775+0000", 2024-09-16T22:16:45.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543776+0000", 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.906e-06 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543783+0000", 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3990000000000001e-06 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543787+0000", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596222+0000", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052435004 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596281+0000", 2024-09-16T22:16:45.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8764999999999997e-05 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596343+0000", 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1493000000000002e-05 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7152 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5079040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:45.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.407610460000001, 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052694566999999998, 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7152 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543795+0000", 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-09-16T22:16:45.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543803+0000", 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2560000000000002e-06 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543810+0000", 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8399999999999997e-06 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543815+0000", 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1420000000000001e-06 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596370+0000", 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052555367999999998 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596433+0000", 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2760000000000002e-05 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596489+0000", 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5819000000000002e-05 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7154 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5115904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.407582459, 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052806530999999997, 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7154 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543823+0000", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.27e-07 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543825+0000", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543831+0000", 2024-09-16T22:16:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543836+0000", 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596513+0000", 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052676730999999997 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596572+0000", 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9531999999999999e-05 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596629+0000", 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6536999999999998e-05 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7155 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5152768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.407561146999999, 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052931824000000002, 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7155 2024-09-16T22:16:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543844+0000", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543846+0000", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0870000000000001e-06 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543853+0000", 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:16:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543858+0000", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596654+0000", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052796211000000003 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596718+0000", 2024-09-16T22:16:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4046000000000003e-05 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596775+0000", 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7497999999999997e-05 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7157 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5189632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.407540153999999, 2024-09-16T22:16:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053064113000000003, 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7157 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548983+0000", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051182490000000001 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548987+0000", 2024-09-16T22:16:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548999+0000", 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599e-05 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.549008+0000", 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999995e-06 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596801+0000", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047793565000000003 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596860+0000", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8761000000000002e-05 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596929+0000", 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8666999999999998e-05 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7229 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5148672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.397488492000001, 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052536309000000003, 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7229 2024-09-16T22:16:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553929+0000", 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2140000000000001e-05 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553936+0000", 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0550000000000002e-06 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553942+0000", 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606177+0000", 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052235072 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606379+0000", 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020166100000000001 2024-09-16T22:16:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606452+0000", 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3753999999999999e-05 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7230 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.397454458999999, 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052682412999999997, 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7230 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557074+0000", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031241379999999998 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557080+0000", 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2549999999999997e-06 2024-09-16T22:16:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557091+0000", 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1121000000000001e-05 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557098+0000", 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3799999999999996e-06 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606482+0000", 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049383972999999998 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606557+0000", 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5189000000000001e-05 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606633+0000", 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5357000000000004e-05 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.043 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.043 INFO:teuthology.orchestra.run.smithi031.stdout:2018 trunc from 0xce06bd7 to 0xb2fed5e 2024-09-16T22:16:45.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2020 read 0x4092c08 thru 0x4094d26 (0x211f bytes) 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2021 writesame 0x10ce303 thru 0x10d8643 data_size 0x1(0xa341 bytes) 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2022 read 0x1565bc6 thru 0x15742f2 (0xe72d bytes) 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2023 write 0xac8ea38 thru 0xac96edb (0x84a4 bytes) 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2024 punch from 0x7b1f4e0 to 0x7b2ddd3, (0xe8f3 bytes) 2024-09-16T22:16:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:2025 write 0xe5b866e thru 0xe5b9620 (0xfb3 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2026 trunc from 0xe5b9621 to 0x86af591 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2027 write 0x636343b thru 0x636c275 (0x8e3b bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2028 trunc from 0x86af591 to 0xcaec21d 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2029 read 0x966a97 thru 0x973bee (0xd158 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2030 punch from 0x28ed550 to 0x28f2694, (0x5144 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2031 read 0x8ba07fe thru 0x8bad4d1 (0xccd4 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2032 write 0xeae05b9 thru 0xeae1e80 (0x18c8 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2033 write 0xb71c4c6 thru 0xb71ca5a (0x595 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2034 read 0x306fefa thru 0x3077f82 (0x8089 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2035 read 0xbc3a0b2 thru 0xbc44544 (0xa493 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2036 read 0xa6ba685 thru 0xa6c3627 (0x8fa3 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2037 punch from 0x712a6a8 to 0x713748a, (0xcde2 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2038 punch from 0x61c3520 to 0x61ca411, (0x6ef1 bytes) 2024-09-16T22:16:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:2039 punch from 0xc200370 to 0xc20e066, (0xdcf6 bytes) 2024-09-16T22:16:45.049 INFO:teuthology.orchestra.run.smithi031.stdout:2040 write 0x16e3b25 thru 0x16e4d40 (0x121c bytes) 2024-09-16T22:16:45.049 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.049 INFO:teuthology.orchestra.run.smithi031.stdout:2043 writesame 0x61428e2 thru 0x61476d9 data_size 0xa(0x4df8 bytes) 2024-09-16T22:16:45.049 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:45.044+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:45.062 INFO:teuthology.orchestra.run.smithi031.stdout:2044 trunc from 0xeae1e81 to 0x186ee18 2024-09-16T22:16:45.093 INFO:teuthology.orchestra.run.smithi031.stdout:2045 write 0xcc0a51e thru 0xcc176dc (0xd1bf bytes) 2024-09-16T22:16:45.096 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.096 INFO:teuthology.orchestra.run.smithi031.stdout:2047 punch from 0x9f2dd10 to 0x9f37673, (0x9963 bytes) 2024-09-16T22:16:45.102 INFO:teuthology.orchestra.run.smithi031.stdout:2048 write 0xd3375fe thru 0xd3388ef (0x12f2 bytes) 2024-09-16T22:16:45.106 INFO:teuthology.orchestra.run.smithi031.stdout:2051 read 0x46e8775 thru 0x46f7edf (0xf76b bytes) 2024-09-16T22:16:45.106 INFO:teuthology.orchestra.run.smithi031.stdout:2052 writesame 0xa564305 thru 0xa5648a3 data_size 0x1(0x59f bytes) 2024-09-16T22:16:45.109 INFO:teuthology.orchestra.run.smithi031.stdout:2053 write 0x55477eb thru 0x5549f80 (0x2796 bytes) 2024-09-16T22:16:45.113 INFO:teuthology.orchestra.run.smithi031.stdout:2054 punch from 0x9daad21 to 0x9db0d59, (0x6038 bytes) 2024-09-16T22:16:45.113 INFO:teuthology.orchestra.run.smithi031.stdout:2055 read 0x503fa39 thru 0x50484db (0x8aa3 bytes) 2024-09-16T22:16:45.127 INFO:teuthology.orchestra.run.smithi031.stdout:2057 trunc from 0xd3388f0 to 0x1103d21 2024-09-16T22:16:45.143 INFO:teuthology.orchestra.run.smithi031.stdout:2058 trunc from 0x1103d21 to 0x6542ab9 2024-09-16T22:16:45.144 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:45.146+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:45.151 INFO:teuthology.orchestra.run.smithi031.stdout:2059 trunc from 0x6542ab9 to 0x9223d28 2024-09-16T22:16:45.156 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.165 INFO:teuthology.orchestra.run.smithi031.stdout:2063 trunc from 0x9223d28 to 0x95b8e7 2024-09-16T22:16:45.174 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:45.174 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.182 INFO:teuthology.orchestra.run.smithi031.stdout:2065 write 0x149b679 thru 0x14a3433 (0x7dbb bytes) 2024-09-16T22:16:45.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:2068 punch from 0x2e2e1c to 0x2f268b, (0xf86f bytes) 2024-09-16T22:16:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:2070 punch from 0x632d4c to 0x63722b, (0x44df bytes) 2024-09-16T22:16:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:2071 punch from 0x127ce6a to 0x1284967, (0x7afd bytes) 2024-09-16T22:16:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.192 INFO:teuthology.orchestra.run.smithi031.stdout:2073 write 0x2904293 thru 0x29067cc (0x253a bytes) 2024-09-16T22:16:45.210 INFO:teuthology.orchestra.run.smithi031.stdout:2074 writesame 0x9c6aed2 thru 0x9c6fb43 data_size 0x26(0x4c72 bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2076 punch from 0x1019be1 to 0x101dc73, (0x4092 bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2077 read 0x7779e0f thru 0x777aab5 (0xca7 bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2078 punch from 0x3c594d1 to 0x3c6152d, (0x805c bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2079 read 0x1699f9b thru 0x16a0d9b (0x6e01 bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2080 read 0x6253485 thru 0x625b808 (0x8384 bytes) 2024-09-16T22:16:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:2081 read 0x3dd4e0 thru 0x3e728b (0x9dac bytes) 2024-09-16T22:16:45.219 INFO:teuthology.orchestra.run.smithi031.stdout:2082 write 0xcdbc542 thru 0xcdc1b90 (0x564f bytes) 2024-09-16T22:16:45.225 INFO:teuthology.orchestra.run.smithi031.stdout:2083 write 0x8ec254d thru 0x8ec8bab (0x665f bytes) 2024-09-16T22:16:45.228 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:45.228 INFO:teuthology.orchestra.run.smithi031.stdout:2085 punch from 0x1efcda5 to 0x1f00c90, (0x3eeb bytes) 2024-09-16T22:16:45.228 INFO:teuthology.orchestra.run.smithi031.stdout:2087 punch from 0x61febed to 0x62023a5, (0x37b8 bytes) 2024-09-16T22:16:45.228 INFO:teuthology.orchestra.run.smithi031.stdout:2089 clone 19 order 18 su 131072 sc 13 2024-09-16T22:16:45.245 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:45.246+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:45.266 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T22:16:45.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T22:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:45.353 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:45.355+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:45.448 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:45.449+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:45.464 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17313 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054412686999999, 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028206102, 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17313 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411396+0000", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411397+0000", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411399+0000", 2024-09-16T22:16:45.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411406+0000", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7270000000000001e-06 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411412+0000", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.331e-06 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439464+0000", 2024-09-16T22:16:45.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028051439000000001 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439551+0000", 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7180000000000002e-05 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439603+0000", 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1613999999999997e-05 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17314 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8437760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054392039, 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028296733000000001, 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17314 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411417+0000", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999998e-07 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411419+0000", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3179999999999999e-06 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411423+0000", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-06 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411426+0000", 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3079999999999999e-06 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439631+0000", 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028204916999999999 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439670+0000", 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8958000000000002e-05 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439714+0000", 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3451000000000001e-05 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17315 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8519680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054378895999999, 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028395437999999999, 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17315 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411430+0000", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411431+0000", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411432+0000", 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0920000000000001e-06 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411435+0000", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411438+0000", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1499999999999999e-06 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439741+0000", 2024-09-16T22:16:45.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028303423000000001 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439784+0000", 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2768000000000001e-05 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439826+0000", 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17316 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8601600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054367880999999, 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028532334999999999, 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17316 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411441+0000", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411442+0000", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7600000000000006e-07 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411445+0000", 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0340000000000001e-06 2024-09-16T22:16:45.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411448+0000", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6290000000000002e-06 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439858+0000", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028409997999999999 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439898+0000", 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9672999999999998e-05 2024-09-16T22:16:45.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.439974+0000", 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5795000000000002e-05 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17317 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8683520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054357289, 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028635106, 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17317 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411452+0000", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411453+0000", 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411456+0000", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9359999999999999e-06 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411459+0000", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440000+0000", 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028541183000000001 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440046+0000", 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5833e-05 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440087+0000", 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0679999999999997e-05 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17318 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8765440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054346666000001, 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028735130000000001, 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17318 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411462+0000", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411463+0000", 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2399999999999999e-07 2024-09-16T22:16:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411464+0000", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0950000000000001e-06 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411467+0000", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-06 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411469+0000", 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-16T22:16:45.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440114+0000", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028645204 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440157+0000", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2110999999999997e-05 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440198+0000", 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1032000000000003e-05 2024-09-16T22:16:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17319 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054336748000001, 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028838682000000001, 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17319 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411472+0000", 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411473+0000", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2700000000000001e-07 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411474+0000", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411477+0000", 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9660000000000001e-06 2024-09-16T22:16:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411479+0000", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4159999999999998e-06 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440225+0000", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028746163000000002 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440271+0000", 2024-09-16T22:16:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5346999999999998e-05 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440311+0000", 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0500000000000002e-05 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17320 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 8929280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:45.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054326999000001, 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028955774, 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17320 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:45.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411482+0000", 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411484+0000", 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411486+0000", 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.915e-06 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411489+0000", 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440346+0000", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028857256000000001 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440397+0000", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1091000000000003e-05 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440438+0000", 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0717999999999999e-05 2024-09-16T22:16:45.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17321 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054316862, 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029072324, 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17321 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411492+0000", 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411493+0000", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4300000000000003e-07 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411494+0000", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.561e-06 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411500+0000", 2024-09-16T22:16:45.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6180000000000003e-06 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411506+0000", 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5779999999999998e-06 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440465+0000", 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028959748 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440504+0000", 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8214999999999999e-05 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440565+0000", 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1061000000000004e-05 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17322 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9093120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054297010999999, 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029196468, 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17322 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411512+0000", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411513+0000", 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-16T22:16:45.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411514+0000", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.894e-06 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411520+0000", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411526+0000", 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5249999999999996e-06 2024-09-16T22:16:45.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440597+0000", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029070833000000001 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440665+0000", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8402e-05 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440709+0000", 2024-09-16T22:16:45.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3424000000000003e-05 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17323 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054277093, 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029288971, 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17323 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411532+0000", 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411534+0000", 2024-09-16T22:16:45.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9209999999999999e-06 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411540+0000", 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0070000000000003e-06 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411545+0000", 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5519999999999998e-06 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440738+0000", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029193447000000001 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440779+0000", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1047999999999997e-05 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440821+0000", 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1621e-05 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17324 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.054258184, 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029399292, 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17324 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411551+0000", 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3099999999999998e-07 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411560+0000", 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1330000000000005e-06 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411567+0000", 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411572+0000", 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440849+0000", 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029276555999999999 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440901+0000", 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2608000000000002e-05 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440950+0000", 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8971000000000003e-05 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17325 3.16 3:68190fba:::rbd_data.11cdc6143ee9.0000000000000003:a [read 9338880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.05422892, 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029483256999999999, 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17325 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.411580+0000", 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417013+0000", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005432727 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417019+0000", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5599999999999999e-06 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417028+0000", 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-16T22:16:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417034+0000", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.440980+0000", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02394655 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.441022+0000", 2024-09-16T22:16:45.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2290000000000003e-05 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.441063+0000", 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1032000000000003e-05 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17358 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048369588, 2024-09-16T22:16:45.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028089057000000001, 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17358 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417440+0000", 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.41e-07 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417441+0000", 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0410000000000001e-06 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417445+0000", 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417448+0000", 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5330000000000001e-06 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445518+0000", 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028070674 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445529+0000", 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0319000000000001e-05 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17359 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048358444, 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028100824, 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17359 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417451+0000", 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-16T22:16:45.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417452+0000", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417455+0000", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.022e-06 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417457+0000", 2024-09-16T22:16:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445547+0000", 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028089571000000001 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445551+0000", 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5430000000000004e-06 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17360 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048346024000001, 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028114694999999999, 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17360 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417463+0000", 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:16:45.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417465+0000", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.181e-06 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417468+0000", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1470000000000001e-06 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417470+0000", 2024-09-16T22:16:45.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445569+0000", 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028098483 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445578+0000", 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0950000000000002e-06 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17361 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048335455, 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028118417, 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17361 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417474+0000", 2024-09-16T22:16:45.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.28e-07 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417475+0000", 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417478+0000", 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417480+0000", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445588+0000", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028107783000000001 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445592+0000", 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0860000000000004e-06 2024-09-16T22:16:45.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17362 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048315369000001, 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134688000000001, 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17362 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417494+0000", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7399999999999999e-07 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417496+0000", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4980000000000001e-06 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417500+0000", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5399999999999997e-06 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417503+0000", 2024-09-16T22:16:45.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445620+0000", 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116306000000001 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445628+0000", 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6479999999999992e-06 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17363 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048301756000001, 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028170649999999998, 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17363 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417507+0000", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417508+0000", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417509+0000", 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.119e-06 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417512+0000", 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0019999999999998e-06 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417514+0000", 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4040000000000002e-06 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445669+0000", 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028154708000000001 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445678+0000", 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1579999999999994e-06 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:17364 3.16 3.5df09816 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.048291879000001, 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028207459000000001, 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 17364 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:45.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417517+0000", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417518+0000", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417519+0000", 2024-09-16T22:16:45.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.04e-06 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417521+0000", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8600000000000001e-06 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.417524+0000", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445714+0000", 2024-09-16T22:16:45.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028190336999999999 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:32.445725+0000", 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0451999999999999e-05 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:45.552 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:45.554 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:45.555+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:45.654 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:45.655+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:45.701 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:45.701 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:45.793 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:45.795+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:45.856 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:45.856+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:45.856 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:45.857 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:45.857 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:45.857 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:45.857 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:45.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:45.957 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:45.957+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.587700208999998, 2024-09-16T22:16:46.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:46.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:46.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31446 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496536+0000", 2024-09-16T22:16:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5186547959999999, 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020250475, 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31446 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496536+0000", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496536+0000", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496536+0000", 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496538+0000", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496545+0000", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8460000000000002e-06 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496550+0000", 2024-09-16T22:16:46.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0050000000000004e-06 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516663+0000", 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020113276999999999 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516728+0000", 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4838999999999994e-05 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516786+0000", 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8233000000000003e-05 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31471 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5185735280000001, 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020335977000000002, 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31471 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496624+0000", 2024-09-16T22:16:46.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8589999999999999e-06 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496642+0000", 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142e-05 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496648+0000", 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516812+0000", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020163871 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516873+0000", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0956999999999999e-05 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516953+0000", 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9983999999999999e-05 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31473 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5185332389999999, 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020442202, 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31473 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496658+0000", 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496661+0000", 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-06 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496668+0000", 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3259999999999998e-06 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496673+0000", 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4040000000000003e-06 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516979+0000", 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020305882000000001 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517041+0000", 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1307000000000001e-05 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517099+0000", 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8817000000000003e-05 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31475 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5185105830000003, 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021013756000000001, 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31475 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496682+0000", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496689+0000", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.652e-06 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496694+0000", 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-16T22:16:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517567+0000", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020872472 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517631+0000", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3998000000000006e-05 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517694+0000", 2024-09-16T22:16:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3106000000000002e-05 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31567 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5184286470000004, 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02048005, 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31567 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496765+0000", 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.219e-06 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496772+0000", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496777+0000", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8790000000000002e-06 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517122+0000", 2024-09-16T22:16:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020345268999999999 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517182+0000", 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0195000000000002e-05 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517242+0000", 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9512999999999998e-05 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31568 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5184071440000002, 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020607819999999999, 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31568 2024-09-16T22:16:46.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496784+0000", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496786+0000", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496801+0000", 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5517000000000001e-05 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496807+0000", 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7060000000000001e-06 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517265+0000", 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020458418999999999 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517329+0000", 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3967999999999995e-05 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517391+0000", 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1914000000000004e-05 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31569 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5183755589999999, 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020725204000000001, 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31569 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496818+0000", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.464e-06 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496826+0000", 2024-09-16T22:16:46.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8159999999999994e-06 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496831+0000", 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489999999999996e-06 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517414+0000", 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020582975999999999 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517480+0000", 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5260999999999997e-05 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517540+0000", 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0537e-05 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31575 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:46.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5183513470000003, 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021677216999999999, 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31575 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496840+0000", 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496842+0000", 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5110000000000002e-06 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496849+0000", 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496854+0000", 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517723+0000", 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020868653000000001 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517796+0000", 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399000000000007e-05 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518516+0000", 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072027500000000004 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31576 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5183293600000001, 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021185559999999999, 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31576 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496862+0000", 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496864+0000", 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3549999999999999e-06 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496870+0000", 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119999999999998e-06 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496875+0000", 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517895+0000", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021019867000000001 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517972+0000", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6711000000000005e-05 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518047+0000", 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.483e-05 2024-09-16T22:16:46.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31585 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.51830862, 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021312514000000001, 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31585 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496884+0000", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496890+0000", 2024-09-16T22:16:46.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1829999999999999e-06 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496896+0000", 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0660000000000003e-06 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518071+0000", 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021175678 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518134+0000", 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3263999999999996e-05 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518194+0000", 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9871999999999999e-05 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31586 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5182876499999995, 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021438353, 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31586 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:46.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496912+0000", 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5879999999999998e-06 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496925+0000", 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3648999999999999e-05 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496931+0000", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518219+0000", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021288517 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518279+0000", 2024-09-16T22:16:46.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0145000000000001e-05 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518341+0000", 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1717000000000006e-05 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31593 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5182512179999996, 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021611253, 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31593 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496940+0000", 2024-09-16T22:16:46.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496943+0000", 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1109999999999999e-06 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496949+0000", 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4699999999999999e-06 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496954+0000", 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000001e-06 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518371+0000", 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021416918 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518442+0000", 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1662000000000006e-05 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518550+0000", 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108034 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31594 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5182298330000004, 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021755911999999999, 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31594 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496963+0000", 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496969+0000", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496975+0000", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.288e-06 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518576+0000", 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021600891000000001 2024-09-16T22:16:46.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518640+0000", 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3905000000000006e-05 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518717+0000", 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31595 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5182085120000002, 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021907319000000001, 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31595 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496985+0000", 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8849999999999999e-06 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496991+0000", 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2759999999999997e-06 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496997+0000", 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518748+0000", 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021751047999999999 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518811+0000", 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518889+0000", 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8641999999999997e-05 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31603 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5181872419999998, 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022076842999999999, 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31603 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497004+0000", 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497006+0000", 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0549999999999998e-06 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497011+0000", 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5860000000000004e-06 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497016+0000", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4800000000000003e-06 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518950+0000", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021934694000000001 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519017+0000", 2024-09-16T22:16:46.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6538000000000006e-05 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519080+0000", 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31604 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:46.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5181686210000001, 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022218702, 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31604 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:46.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497025+0000", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3259999999999998e-06 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497031+0000", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8370000000000002e-06 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497035+0000", 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519107+0000", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022072692000000001 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519172+0000", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4672999999999996e-05 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519241+0000", 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8367000000000005e-05 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31605 3.11 3.538ddd91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5181491129999998, 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022265193999999999, 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31605 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497042+0000", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497052+0000", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0073e-05 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497059+0000", 2024-09-16T22:16:46.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.007e-06 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497064+0000", 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6559999999999997e-06 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519299+0000", 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022234296000000001 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519307+0000", 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7500000000000003e-06 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31627 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5178826919999997, 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021960872999999999, 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31627 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497310+0000", 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9980000000000002e-06 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497316+0000", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497321+0000", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6909999999999996e-06 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519262+0000", 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021940867999999999 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519269+0000", 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6590000000000001e-06 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31628 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5178410380000003, 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021947498999999999, 2024-09-16T22:16:46.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31628 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497350+0000", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497352+0000", 2024-09-16T22:16:46.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4679999999999999e-06 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497359+0000", 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497365+0000", 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.327e-06 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:46.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519290+0000", 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021925153999999999 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519297+0000", 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1489999999999997e-06 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:46.108 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:46.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:46.110 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:46.110+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:46.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:46.210 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:46.212+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:46.219 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:46.219 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:46.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:46.276 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T22:16:46.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:46.279 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:46.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T22:16:46.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:46.311 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:46.312+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:46.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:46.326 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:46.336 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:46.337 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:46.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:46.347 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:46.347 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:46.364 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:46.364 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:46.382 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:46.382 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:46.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:16:46.399 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:46.417 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:46.417 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:46.430 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:46.431+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:46.472 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T22:16:46.497 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:46.497 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:46.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:46.513 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:46.515+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:46.517 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.517 INFO:teuthology.orchestra.run.smithi031.stdout:2091 write 0x4ebf98e thru 0x4ecb3a3 (0xba16 bytes) 2024-09-16T22:16:46.533 INFO:teuthology.orchestra.run.smithi031.stdout:2092 write 0xe958c6e thru 0xe95b344 (0x26d7 bytes) 2024-09-16T22:16:46.537 INFO:teuthology.orchestra.run.smithi031.stdout:2093 read 0x43c7385 thru 0x43cdfcb (0x6c47 bytes) 2024-09-16T22:16:46.537 INFO:teuthology.orchestra.run.smithi031.stdout:2094 read 0x4c4f93d thru 0x4c56246 (0x690a bytes) 2024-09-16T22:16:46.554 INFO:teuthology.orchestra.run.smithi031.stdout:2095 trunc from 0xe95b345 to 0x1a7f92c 2024-09-16T22:16:46.571 INFO:teuthology.orchestra.run.smithi031.stdout:2096 read 0x196bb9d thru 0x19738e6 (0x7d4a bytes) 2024-09-16T22:16:46.580 INFO:teuthology.orchestra.run.smithi031.stdout:2097 write 0x4b8ffb7 thru 0x4b99e5c (0x9ea6 bytes) 2024-09-16T22:16:46.600 INFO:teuthology.orchestra.run.smithi031.stdout:2098 write 0xe934cd6 thru 0xe93ee68 (0xa193 bytes) 2024-09-16T22:16:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:2100 punch from 0x4b6e35c to 0x4b78697, (0xa33b bytes) 2024-09-16T22:16:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:2102 read 0x37a13e thru 0x388852 (0xe715 bytes) 2024-09-16T22:16:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:2103 write 0x5285314 thru 0x52892ac (0x3f99 bytes) 2024-09-16T22:16:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:2104 read 0x49b5b70 thru 0x49c48d0 (0xed61 bytes) 2024-09-16T22:16:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:2105 punch from 0xc02b6f4 to 0xc0387a8, (0xd0b4 bytes) 2024-09-16T22:16:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:2106 punch from 0xe146808 to 0xe155c4e, (0xf446 bytes) 2024-09-16T22:16:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:2108 write 0xe418457 thru 0xe41e863 (0x640d bytes) 2024-09-16T22:16:46.610 INFO:teuthology.orchestra.run.smithi031.stdout:2111 punch from 0x8b5d603 to 0x8b673fa, (0x9df7 bytes) 2024-09-16T22:16:46.610 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.614 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:46.615+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:46.623 INFO:teuthology.orchestra.run.smithi031.stdout:2114 trunc from 0xe93ee69 to 0x315e6f3 2024-09-16T22:16:46.640 INFO:teuthology.orchestra.run.smithi031.stdout:2115 writesame 0x3514dc8 thru 0x3524b84 data_size 0x1(0xfdbd bytes) 2024-09-16T22:16:46.641 INFO:teuthology.orchestra.run.smithi031.stdout:2117 read 0x2dc867d thru 0x2dc93a9 (0xd2d bytes) 2024-09-16T22:16:46.641 INFO:teuthology.orchestra.run.smithi031.stdout:2118 write 0x25c7fe1 thru 0x25ce214 (0x6234 bytes) 2024-09-16T22:16:46.652 INFO:teuthology.orchestra.run.smithi031.stdout:2124 trunc from 0x3524b85 to 0xabc585d 2024-09-16T22:16:46.657 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:46.657 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:46.664 INFO:teuthology.orchestra.run.smithi031.stdout:2125 write 0xb090fc6 thru 0xb09441b (0x3456 bytes) 2024-09-16T22:16:46.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:46.668 INFO:teuthology.orchestra.run.smithi031.stdout:2126 writesame 0x7e7f7c7 thru 0x7e8df83 data_size 0x4b(0xe7bd bytes) 2024-09-16T22:16:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:2127 read 0xa455e60 thru 0xa45897d (0x2b1e bytes) 2024-09-16T22:16:46.675 INFO:teuthology.orchestra.run.smithi031.stdout:2128 trunc from 0xb09441c to 0xc1f066c 2024-09-16T22:16:46.681 INFO:teuthology.orchestra.run.smithi031.stdout:2129 write 0x9c15675 thru 0x9c1c159 (0x6ae5 bytes) 2024-09-16T22:16:46.684 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.684 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.684 INFO:teuthology.orchestra.run.smithi031.stdout:2132 punch from 0x6b2e888 to 0x6b3c4f5, (0xdc6d bytes) 2024-09-16T22:16:46.684 INFO:teuthology.orchestra.run.smithi031.stdout:2136 trunc from 0xc1f066c to 0xc67b667 2024-09-16T22:16:46.689 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.689 INFO:teuthology.orchestra.run.smithi031.stdout:2138 write 0x163e90f thru 0x1645d33 (0x7425 bytes) 2024-09-16T22:16:46.696 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.696 INFO:teuthology.orchestra.run.smithi031.stdout:2140 punch from 0x783832a to 0x78452a2, (0xcf78 bytes) 2024-09-16T22:16:46.696 INFO:teuthology.orchestra.run.smithi031.stdout:2141 read 0x32ece9c thru 0x32f57b9 (0x891e bytes) 2024-09-16T22:16:46.696 INFO:teuthology.orchestra.run.smithi031.stdout:2142 read 0xc95e86 thru 0xc9f7b6 (0x9931 bytes) 2024-09-16T22:16:46.702 INFO:teuthology.orchestra.run.smithi031.stdout:2144 writesame 0xd50e3c8 thru 0xd51d6a3 data_size 0x9d(0xf2dc bytes) 2024-09-16T22:16:46.702 INFO:teuthology.orchestra.run.smithi031.stdout:2145 read 0x2908b46 thru 0x29117df (0x8c9a bytes) 2024-09-16T22:16:46.702 INFO:teuthology.orchestra.run.smithi031.stdout:2146 punch from 0xbd58805 to 0xbd5f98c, (0x7187 bytes) 2024-09-16T22:16:46.702 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.703 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.716 INFO:teuthology.orchestra.run.smithi031.stdout:2149 trunc from 0xd51d6a4 to 0x11f2bcd 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:2151 punch from 0x70cad to 0x76ccc, (0x601f bytes) 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:2154 punch from 0x690c to 0x12883, (0xbf77 bytes) 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:2156 punch from 0x10ed5ed to 0x10f00bd, (0x2ad0 bytes) 2024-09-16T22:16:46.744 INFO:teuthology.orchestra.run.smithi031.stdout:2157 read 0x10c2454 thru 0x10c5143 (0x2cf0 bytes) 2024-09-16T22:16:46.750 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:46.751+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:46.763 INFO:teuthology.orchestra.run.smithi031.stdout:2158 writesame 0xd404de4 thru 0xd40aa03 data_size 0x58(0x5c20 bytes) 2024-09-16T22:16:46.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.764 INFO:teuthology.orchestra.run.smithi031.stdout:2160 read 0xd012887 thru 0xd02215b (0xf8d5 bytes) 2024-09-16T22:16:46.764 INFO:teuthology.orchestra.run.smithi031.stdout:2161 punch from 0x49456c9 to 0x495050d, (0xae44 bytes) 2024-09-16T22:16:46.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.764 INFO:teuthology.orchestra.run.smithi031.stdout:2164 writesame 0xcc369a8 thru 0xcc3852e data_size 0x1d(0x1b87 bytes) 2024-09-16T22:16:46.766 INFO:teuthology.orchestra.run.smithi031.stdout:2166 read 0xd0b1ba0 thru 0xd0b7b94 (0x5ff5 bytes) 2024-09-16T22:16:46.775 INFO:teuthology.orchestra.run.smithi031.stdout:2167 trunc from 0xd40aa04 to 0x58944f6 2024-09-16T22:16:46.784 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.794 INFO:teuthology.orchestra.run.smithi031.stdout:2170 write 0xa01fed6 thru 0xa02250d (0x2638 bytes) 2024-09-16T22:16:46.798 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.798 INFO:teuthology.orchestra.run.smithi031.stdout:2172 write 0x893d052 thru 0x893ffb6 (0x2f65 bytes) 2024-09-16T22:16:46.800 INFO:teuthology.orchestra.run.smithi031.stdout:2173 read 0x33cf49e thru 0x33dc10e (0xcc71 bytes) 2024-09-16T22:16:46.801 INFO:teuthology.orchestra.run.smithi031.stdout:2174 punch from 0x769986c to 0x76a4740, (0xaed4 bytes) 2024-09-16T22:16:46.801 INFO:teuthology.orchestra.run.smithi031.stdout:2175 writesame 0x756215f thru 0x756ab50 data_size 0x1(0x89f2 bytes) 2024-09-16T22:16:46.816 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:46.817+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:46.818 INFO:teuthology.orchestra.run.smithi031.stdout:2176 read 0x2a20ad6 thru 0x2a20e7b (0x3a6 bytes) 2024-09-16T22:16:46.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:46.828 INFO:teuthology.orchestra.run.smithi031.stdout:2177 trunc from 0xa02250e to 0x18ae735 2024-09-16T22:16:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:2180 read 0xe5328c thru 0xe5b8db (0x8650 bytes) 2024-09-16T22:16:46.854 INFO:teuthology.orchestra.run.smithi031.stdout:2181 write 0xa777118 thru 0xa786a9f (0xf988 bytes) 2024-09-16T22:16:46.860 INFO:teuthology.orchestra.run.smithi031.stdout:2182 write 0x109f210 thru 0x10a26b2 (0x34a3 bytes) 2024-09-16T22:16:46.870 INFO:teuthology.orchestra.run.smithi031.stdout:2184 writesame 0xbe6ae59 thru 0xbe6f8de data_size 0x1(0x4a86 bytes) 2024-09-16T22:16:46.876 INFO:teuthology.orchestra.run.smithi031.stdout:2185 write 0xcc58026 thru 0xcc65f11 (0xdeec bytes) 2024-09-16T22:16:46.881 INFO:teuthology.orchestra.run.smithi031.stdout:2187 punch from 0xc02901b to 0xc0315a9, (0x858e bytes) 2024-09-16T22:16:46.882 INFO:teuthology.orchestra.run.smithi031.stdout:2188 read 0x6e8eba9 thru 0x6e9b5c3 (0xca1b bytes) 2024-09-16T22:16:46.882 INFO:teuthology.orchestra.run.smithi031.stdout:2189 writesame 0x84644d7 thru 0x846a5d2 data_size 0x6(0x60fc bytes) 2024-09-16T22:16:46.886 INFO:teuthology.orchestra.run.smithi031.stdout:2190 read 0x1b29908 thru 0x1b307b7 (0x6eb0 bytes) 2024-09-16T22:16:46.886 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.886 INFO:teuthology.orchestra.run.smithi031.stdout:2193 read 0xb61c68d thru 0xb6262d3 (0x9c47 bytes) 2024-09-16T22:16:46.886 INFO:teuthology.orchestra.run.smithi031.stdout:2194 write 0x146d4c4 thru 0x14774cc (0xa009 bytes) 2024-09-16T22:16:46.890 INFO:teuthology.orchestra.run.smithi031.stdout:2196 punch from 0xb882a99 to 0xb88ab1e, (0x8085 bytes) 2024-09-16T22:16:46.899 INFO:teuthology.orchestra.run.smithi031.stdout:2198 trunc from 0xcc65f12 to 0x49fb636 2024-09-16T22:16:46.910 INFO:teuthology.orchestra.run.smithi031.stdout:2199 write 0x3164f54 thru 0x3169954 (0x4a01 bytes) 2024-09-16T22:16:46.916 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:46.917+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:46.920 INFO:teuthology.orchestra.run.smithi031.stdout:2201 write 0x500e570 thru 0x501bf81 (0xda12 bytes) 2024-09-16T22:16:46.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.923 INFO:teuthology.orchestra.run.smithi031.stdout:2204 read 0x2cb94b0 thru 0x2cbedbc (0x590d bytes) 2024-09-16T22:16:46.923 INFO:teuthology.orchestra.run.smithi031.stdout:2205 read 0x4ff1dd1 thru 0x4ff9275 (0x74a5 bytes) 2024-09-16T22:16:46.924 INFO:teuthology.orchestra.run.smithi031.stdout:2207 punch from 0x419ac42 to 0x419fdbf, (0x517d bytes) 2024-09-16T22:16:46.939 INFO:teuthology.orchestra.run.smithi031.stdout:2208 write 0xe18c21e thru 0xe192a9d (0x6880 bytes) 2024-09-16T22:16:46.942 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:46.942 INFO:teuthology.orchestra.run.smithi031.stdout:2210 writesame 0x4cb994e thru 0x4cb9e10 data_size 0x1(0x4c3 bytes) 2024-09-16T22:16:46.945 INFO:teuthology.orchestra.run.smithi031.stdout:2211 punch from 0x8c1144b to 0x8c1716f, (0x5d24 bytes) 2024-09-16T22:16:46.960 INFO:teuthology.orchestra.run.smithi031.stdout:2213 trunc from 0xe192a9e to 0x4ddc5b 2024-09-16T22:16:46.974 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:46.974 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:46.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:46.993 INFO:teuthology.orchestra.run.smithi031.stdout:2214 write 0x2a2cc18 thru 0x2a354f6 (0x88df bytes) 2024-09-16T22:16:47.010 INFO:teuthology.orchestra.run.smithi031.stdout:2215 write 0xa0bcdf1 thru 0xa0be970 (0x1b80 bytes) 2024-09-16T22:16:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:2218 write 0x9958db thru 0x9a06fd (0xae23 bytes) 2024-09-16T22:16:47.017 INFO:teuthology.orchestra.run.smithi031.stdout:2220 write 0x97b721b thru 0x97bcc14 (0x59fa bytes) 2024-09-16T22:16:47.017 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:47.019+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:2221 trunc from 0xa0be971 to 0x6d1955 2024-09-16T22:16:47.061 INFO:teuthology.orchestra.run.smithi031.stdout:2222 write 0x5117f92 thru 0x511d25a (0x52c9 bytes) 2024-09-16T22:16:47.068 INFO:teuthology.orchestra.run.smithi031.stdout:2224 trunc from 0x511d25b to 0x1852f0c 2024-09-16T22:16:47.077 INFO:teuthology.orchestra.run.smithi031.stdout:2225 punch from 0xca700f to 0xcaec13, (0x7c04 bytes) 2024-09-16T22:16:47.085 INFO:teuthology.orchestra.run.smithi031.stdout:2227 write 0x459fb28 thru 0x45ac18b (0xc664 bytes) 2024-09-16T22:16:47.103 INFO:teuthology.orchestra.run.smithi031.stdout:2230 write 0xa0ca9e6 thru 0xa0d7643 (0xcc5e bytes) 2024-09-16T22:16:47.106 INFO:teuthology.orchestra.run.smithi031.stdout:2232 read 0x4871106 thru 0x4873341 (0x223c bytes) 2024-09-16T22:16:47.106 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.106 INFO:teuthology.orchestra.run.smithi031.stdout:2234 read 0x58d2c5 thru 0x58e9e2 (0x171e bytes) 2024-09-16T22:16:47.106 INFO:teuthology.orchestra.run.smithi031.stdout:2235 read 0x4c1b3bb thru 0x4c1e58c (0x31d2 bytes) 2024-09-16T22:16:47.114 INFO:teuthology.orchestra.run.smithi031.stdout:2236 write 0xc37c348 thru 0xc38b03b (0xecf4 bytes) 2024-09-16T22:16:47.118 INFO:teuthology.orchestra.run.smithi031.stdout:2237 write 0x1d399b0 thru 0x1d43758 (0x9da9 bytes) 2024-09-16T22:16:47.118 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:47.119+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:47.125 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.125 INFO:teuthology.orchestra.run.smithi031.stdout:2239 write 0x151c06b thru 0x151db44 (0x1ada bytes) 2024-09-16T22:16:47.130 INFO:teuthology.orchestra.run.smithi031.stdout:2240 punch from 0xa66c184 to 0xa671985, (0x5801 bytes) 2024-09-16T22:16:47.130 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.131 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.131 INFO:teuthology.orchestra.run.smithi031.stdout:2243 write 0x3ef8cf thru 0x3fa5a3 (0xacd5 bytes) 2024-09-16T22:16:47.132 INFO:teuthology.orchestra.run.smithi031.stdout:2244 punch from 0x8690661 to 0x86972da, (0x6c79 bytes) 2024-09-16T22:16:47.132 INFO:teuthology.orchestra.run.smithi031.stdout:2246 clone 20 order 18 su 2048 sc 4 2024-09-16T22:16:47.132 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.133 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:47.225 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:47.226+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T22:16:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:16:47.285 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.285 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:47.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T22:16:47.338 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:47.340+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:47.340 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone19 intact 2024-09-16T22:16:47.420 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:47.421+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:47.438 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.438 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:47.474 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T22:16:47.522 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:47.523+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:47.597 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.597 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:47.622 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:47.623+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:47.707 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.707 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:47.718 INFO:teuthology.orchestra.run.smithi031.stdout:2247 punch from 0x42bb7d8 to 0x42c7bae, (0xc3d6 bytes) 2024-09-16T22:16:47.722 INFO:teuthology.orchestra.run.smithi031.stdout:2248 write 0x4e71f84 thru 0x4e72dc8 (0xe45 bytes) 2024-09-16T22:16:47.728 INFO:teuthology.orchestra.run.smithi031.stdout:2249 write 0x6b9c063 thru 0x6ba6267 (0xa205 bytes) 2024-09-16T22:16:47.734 INFO:teuthology.orchestra.run.smithi031.stdout:2250 read 0x95d0cd8 thru 0x95d19b4 (0xcdd bytes) 2024-09-16T22:16:47.734 INFO:teuthology.orchestra.run.smithi031.stdout:2251 write 0x5cbe77d thru 0x5ccadce (0xc652 bytes) 2024-09-16T22:16:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:2252 write 0xc8f384e thru 0xc8f5d47 (0x24fa bytes) 2024-09-16T22:16:47.751 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.751 INFO:teuthology.orchestra.run.smithi031.stdout:2254 read 0xa546aef thru 0xa54abfa (0x410c bytes) 2024-09-16T22:16:47.757 INFO:teuthology.orchestra.run.smithi031.stdout:2255 trunc from 0xc8f5d48 to 0x798cedc 2024-09-16T22:16:47.759 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:47.760+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:47.777 INFO:teuthology.orchestra.run.smithi031.stdout:2257 write 0x4ba9328 thru 0x4bb658f (0xd268 bytes) 2024-09-16T22:16:47.783 INFO:teuthology.orchestra.run.smithi031.stdout:2258 read 0x476b3b4 thru 0x477482a (0x9477 bytes) 2024-09-16T22:16:47.783 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.783 INFO:teuthology.orchestra.run.smithi031.stdout:2261 write 0x49fb00b thru 0x49fbc00 (0xbf6 bytes) 2024-09-16T22:16:47.787 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.789 INFO:teuthology.orchestra.run.smithi031.stdout:2267 trunc from 0x798cedc to 0x9b4da48 2024-09-16T22:16:47.794 INFO:teuthology.orchestra.run.smithi031.stdout:2268 write 0x79e60c5 thru 0x79e68eb (0x827 bytes) 2024-09-16T22:16:47.801 INFO:teuthology.orchestra.run.smithi031.stdout:2271 write 0xa6cd032 thru 0xa6d3b97 (0x6b66 bytes) 2024-09-16T22:16:47.807 INFO:teuthology.orchestra.run.smithi031.stdout:2273 punch from 0x262cc6c to 0x262f6e2, (0x2a76 bytes) 2024-09-16T22:16:47.807 INFO:teuthology.orchestra.run.smithi031.stdout:2274 write 0xa18bbd8 thru 0xa192a61 (0x6e8a bytes) 2024-09-16T22:16:47.817 INFO:teuthology.orchestra.run.smithi031.stdout:2275 write 0xa7204ef thru 0xa72c9c0 (0xc4d2 bytes) 2024-09-16T22:16:47.821 INFO:teuthology.orchestra.run.smithi031.stdout:2278 write 0x8358e2c thru 0x835e4a1 (0x5676 bytes) 2024-09-16T22:16:47.822 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:47.824+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:2279 punch from 0x2596b0a to 0x2597a4a, (0xf40 bytes) 2024-09-16T22:16:47.829 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.830 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.833 INFO:teuthology.orchestra.run.smithi031.stdout:2280 write 0xbabbf8d thru 0xbacbec8 (0xff3c bytes) 2024-09-16T22:16:47.838 INFO:teuthology.orchestra.run.smithi031.stdout:2281 punch from 0x406557d to 0x407529f, (0xfd22 bytes) 2024-09-16T22:16:47.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:47.846 INFO:teuthology.orchestra.run.smithi031.stdout:2284 write 0xe12d57b thru 0xe12da01 (0x487 bytes) 2024-09-16T22:16:47.852 INFO:teuthology.orchestra.run.smithi031.stdout:2285 write 0xd0705b6 thru 0xd07e942 (0xe38d bytes) 2024-09-16T22:16:47.861 INFO:teuthology.orchestra.run.smithi031.stdout:2286 writesame 0x53ddcf3 thru 0x53de987 data_size 0x1(0xc95 bytes) 2024-09-16T22:16:47.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.866 INFO:teuthology.orchestra.run.smithi031.stdout:2289 read 0x25a0d7f thru 0x25a6449 (0x56cb bytes) 2024-09-16T22:16:47.866 INFO:teuthology.orchestra.run.smithi031.stdout:2290 writesame 0xa51d6c5 thru 0xa521666 data_size 0x2d(0x3fa2 bytes) 2024-09-16T22:16:47.873 INFO:teuthology.orchestra.run.smithi031.stdout:2291 punch from 0xb856313 to 0xb85e195, (0x7e82 bytes) 2024-09-16T22:16:47.873 INFO:teuthology.orchestra.run.smithi031.stdout:2292 writesame 0xa18fda9 thru 0xa197188 data_size 0x20(0x73e0 bytes) 2024-09-16T22:16:47.874 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.878 INFO:teuthology.orchestra.run.smithi031.stdout:2294 trunc from 0xe12da02 to 0xaf01e52 2024-09-16T22:16:47.889 INFO:teuthology.orchestra.run.smithi031.stdout:2295 write 0x1019971 thru 0x1025192 (0xb822 bytes) 2024-09-16T22:16:47.898 INFO:teuthology.orchestra.run.smithi031.stdout:2296 writesame 0x33dcc3c thru 0x33e86a3 data_size 0x14(0xba68 bytes) 2024-09-16T22:16:47.904 INFO:teuthology.orchestra.run.smithi031.stdout:2298 writesame 0x2180f72 thru 0x2180fcf data_size 0x1(0x5e bytes) 2024-09-16T22:16:47.908 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:47.908 INFO:teuthology.orchestra.run.smithi031.stdout:2301 clone 21 order 24 su 524288 sc 8 2024-09-16T22:16:47.923 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:47.924+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:47.959 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:47.959 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:47.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:48.024 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:48.025+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:48.115 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:48.115 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:48.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:48.124 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:48.126+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:48.225 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:48.226+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:48.230 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:48.230 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:48.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:48.290 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T22:16:48.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T22:16:48.325 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:48.326+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:48.357 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:48.357 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:48.362 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone20 from 0xaf01e52 (overlap 0x798cedc) to 0x613d8b0 2024-09-16T22:16:48.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:48.426 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:48.427+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:48.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:48.513 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:48.513 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:48.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:48.527 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:48.529+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:48.571 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T22:16:48.591 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:48.612 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:48.628 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:48.629+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:48.644 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:48.661 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:48.666 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7042 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4591616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.135401815000002, 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052761957999999998, 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7042 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531307+0000", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531308+0000", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531315+0000", 2024-09-16T22:16:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1470000000000004e-06 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531323+0000", 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6979999999999998e-06 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531328+0000", 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2619999999999998e-06 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583754+0000", 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052425839000000002 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.583827+0000", 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2933000000000001e-05 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584069+0000", 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024263 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7043 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4628480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.135365909000001, 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052969308, 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7043 2024-09-16T22:16:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531343+0000", 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531344+0000", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531346+0000", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2520000000000002e-06 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531354+0000", 2024-09-16T22:16:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1329999999999999e-06 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531360+0000", 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3380000000000004e-06 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584131+0000", 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052771021000000001 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584233+0000", 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102227 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584313+0000", 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9849000000000001e-05 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7044 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4665344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.135341987, 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053119052999999999, 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7044 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531367+0000", 2024-09-16T22:16:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531703+0000", 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033545599999999998 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531707+0000", 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4959999999999998e-06 2024-09-16T22:16:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531720+0000", 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2830000000000001e-05 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531727+0000", 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1629999999999999e-06 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584342+0000", 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052614840000000003 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584414+0000", 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1876000000000001e-05 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584486+0000", 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2391999999999994e-05 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7045 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4702208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.134962513000001, 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052901939000000002, 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7045 2024-09-16T22:16:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531747+0000", 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531750+0000", 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531758+0000", 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7379999999999994e-06 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531775+0000", 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.685e-05 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584514+0000", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052739923000000001 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584583+0000", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8868999999999994e-05 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584649+0000", 2024-09-16T22:16:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5377999999999996e-05 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7046 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 4739072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.134924718000001, 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053033981000000001, 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.531785+0000", 2024-09-16T22:16:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533754+0000", 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019694040000000001 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533759+0000", 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533775+0000", 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5441999999999999e-05 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.533784+0000", 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.884e-06 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584675+0000", 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050891758000000002 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584750+0000", 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4411000000000003e-05 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.584819+0000", 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8783999999999997e-05 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7055 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5070848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.132667665, 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052727892999999998, 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7055 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534042+0000", 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534044+0000", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.317e-06 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534051+0000", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.212e-06 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534056+0000", 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586519+0000", 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052463981999999999 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586618+0000", 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586770+0000", 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000151368 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7056 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5107712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.132646010999999, 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052802007999999998, 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534063+0000", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534064+0000", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534066+0000", 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-16T22:16:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534072+0000", 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534086+0000", 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3360000000000001e-05 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586726+0000", 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052640575000000002 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586793+0000", 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6611000000000003e-05 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586865+0000", 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2321000000000001e-05 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7057 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5144576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.132613271, 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052927321999999999, 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.534096+0000", 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536968+0000", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028722890000000001 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536973+0000", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.865e-06 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536985+0000", 2024-09-16T22:16:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2196e-05 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.536993+0000", 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7060000000000004e-06 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586893+0000", 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049899678000000003 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.586961+0000", 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8354000000000003e-05 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.587023+0000", 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2234000000000001e-05 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7070 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5623808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.129403045, 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052526675000000002, 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7070 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537306+0000", 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537307+0000", 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537309+0000", 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9429999999999999e-06 2024-09-16T22:16:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537315+0000", 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537320+0000", 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589999999999997e-06 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589660+0000", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052340416000000001 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589747+0000", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6988000000000001e-05 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589833+0000", 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5739e-05 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7071 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5660672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.129383005000001, 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870691999999997, 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7071 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537326+0000", 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537327+0000", 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537329+0000", 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537337+0000", 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1769999999999998e-06 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537342+0000", 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.589872+0000", 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052530514 2024-09-16T22:16:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590113+0000", 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241179 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590197+0000", 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3498000000000006e-05 2024-09-16T22:16:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7072 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5697536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.129357915, 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053032186000000002, 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7072 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537351+0000", 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537352+0000", 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537361+0000", 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1749999999999994e-06 2024-09-16T22:16:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537370+0000", 2024-09-16T22:16:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4560000000000006e-06 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537376+0000", 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6019999999999999e-06 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590237+0000", 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05286101 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590309+0000", 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1928999999999995e-05 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590384+0000", 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4606999999999999e-05 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7073 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5734400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.12932472, 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053168509000000003, 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537385+0000", 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:16:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537387+0000", 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537394+0000", 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7089999999999996e-06 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537399+0000", 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9869999999999999e-06 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590419+0000", 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053019721999999998 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590481+0000", 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2317000000000007e-05 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590553+0000", 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2050000000000003e-05 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7074 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5771264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.129303528000001, 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053310116999999997, 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.537406+0000", 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542944+0000", 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055381600000000003 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542949+0000", 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542961+0000", 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.203e-05 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.542979+0000", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7591000000000001e-05 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590580+0000", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047601516000000003 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590647+0000", 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7044000000000004e-05 2024-09-16T22:16:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.590716+0000", 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8536999999999998e-05 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7150 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5042176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.122935172999998, 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052568581000000003, 2024-09-16T22:16:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7150 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543774+0000", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543775+0000", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543776+0000", 2024-09-16T22:16:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.906e-06 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543783+0000", 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3990000000000001e-06 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543787+0000", 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596222+0000", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052435004 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596281+0000", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8764999999999997e-05 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596343+0000", 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1493000000000002e-05 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7152 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5079040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.122915152000001, 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052694566999999998, 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7152 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543794+0000", 2024-09-16T22:16:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543795+0000", 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543803+0000", 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2560000000000002e-06 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543810+0000", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8399999999999997e-06 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543815+0000", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1420000000000001e-06 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596370+0000", 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052555367999999998 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596433+0000", 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2760000000000002e-05 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596489+0000", 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5819000000000002e-05 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7154 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5115904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.122887151, 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052806530999999997, 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7154 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543822+0000", 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543823+0000", 2024-09-16T22:16:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.27e-07 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543825+0000", 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1260000000000001e-06 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543831+0000", 2024-09-16T22:16:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543836+0000", 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596513+0000", 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052676730999999997 2024-09-16T22:16:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596572+0000", 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9531999999999999e-05 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596629+0000", 2024-09-16T22:16:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6536999999999998e-05 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7155 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5152768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.122865838999999, 2024-09-16T22:16:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052931824000000002, 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7155 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543843+0000", 2024-09-16T22:16:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543844+0000", 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543846+0000", 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0870000000000001e-06 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543853+0000", 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543858+0000", 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596654+0000", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052796211000000003 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596718+0000", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4046000000000003e-05 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596775+0000", 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7497999999999997e-05 2024-09-16T22:16:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7157 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5189632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.122844846, 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053064113000000003, 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7157 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.543864+0000", 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548983+0000", 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051182490000000001 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548987+0000", 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.548999+0000", 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599e-05 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.549008+0000", 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999995e-06 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596801+0000", 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047793565000000003 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596860+0000", 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8761000000000002e-05 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.596929+0000", 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8666999999999998e-05 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7229 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5148672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.112793184000001, 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052536309000000003, 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7229 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553916+0000", 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553929+0000", 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2140000000000001e-05 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553936+0000", 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0550000000000002e-06 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553942+0000", 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606177+0000", 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052235072 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606379+0000", 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020166100000000001 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606452+0000", 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3753999999999999e-05 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:7230 3.1a 3:5d2d0e08:::rbd_data.11cd2cec779f.0000000000000000:6 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.112759150999999, 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052682412999999997, 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7230 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.553950+0000", 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557074+0000", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031241379999999998 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557080+0000", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2549999999999997e-06 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557091+0000", 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1121000000000001e-05 2024-09-16T22:16:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.557098+0000", 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3799999999999996e-06 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606482+0000", 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049383972999999998 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606557+0000", 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5189000000000001e-05 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:25.606633+0000", 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5357000000000004e-05 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:48.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:48.774 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:48.774 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:48.774 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:48.775 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:48.775 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:48.775 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:48.776 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:48.777 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:48.774+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:48.821 INFO:teuthology.orchestra.run.smithi031.stdout:2304 writesame 0xe501078 thru 0xe502d1f data_size 0xe(0x1ca8 bytes) 2024-09-16T22:16:48.825 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:48.825 INFO:teuthology.orchestra.run.smithi031.stdout:2306 read 0x60a8b01 thru 0x60abd84 (0x3284 bytes) 2024-09-16T22:16:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:2307 trunc from 0xe502d20 to 0x926dfc8 2024-09-16T22:16:48.860 INFO:teuthology.orchestra.run.smithi031.stdout:2308 write 0x9d366f7 thru 0x9d36e97 (0x7a1 bytes) 2024-09-16T22:16:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:2310 trunc from 0x9d36e98 to 0x2f326e6 2024-09-16T22:16:48.876 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:48.878+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:48.974 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:48.976+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41676 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.949039+0000", 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037090625000000002, 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037107207000000003, 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41676 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.949039+0000", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.949039+0000", 2024-09-16T22:16:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.949039+0000", 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.949048+0000", 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4790000000000002e-06 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951211+0000", 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021626409999999999 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951225+0000", 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3499000000000001e-05 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41892 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952028+0000", 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034100956000000002, 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034149984000000001, 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41892 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952028+0000", 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952028+0000", 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952029+0000", 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952031+0000", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.272e-06 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952037+0000", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.409e-06 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952043+0000", 2024-09-16T22:16:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41956 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953785+0000", 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032343999999999998, 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032427809000000002, 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41956 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953785+0000", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953785+0000", 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953786+0000", 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953792+0000", 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953799+0000", 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953805+0000", 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4550000000000003e-06 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42068 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960974+0000", 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025155180999999999, 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025285682, 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42068 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960974+0000", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960974+0000", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960974+0000", 2024-09-16T22:16:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960976+0000", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9870000000000002e-06 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960983+0000", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7159999999999997e-06 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960988+0000", 2024-09-16T22:16:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42116 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968068+0000", 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018061451999999999, 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018216992000000001, 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42116 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968068+0000", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968068+0000", 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968068+0000", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968070+0000", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.902e-06 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968076+0000", 2024-09-16T22:16:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968081+0000", 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41677 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951288+0000", 2024-09-16T22:16:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034841506000000001, 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035022040999999997, 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41677 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951288+0000", 2024-09-16T22:16:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951288+0000", 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951289+0000", 2024-09-16T22:16:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951293+0000", 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951307+0000", 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4015999999999999e-05 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951314+0000", 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41893 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952049+0000", 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034080065, 2024-09-16T22:16:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034283780999999999, 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41893 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952049+0000", 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952049+0000", 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952050+0000", 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4700000000000002e-07 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952058+0000", 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4130000000000006e-06 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952065+0000", 2024-09-16T22:16:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1189999999999999e-06 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952070+0000", 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41957 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953814+0000", 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032315635000000002, 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032542725000000002, 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41957 2024-09-16T22:16:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953814+0000", 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953814+0000", 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953814+0000", 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953816+0000", 2024-09-16T22:16:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953823+0000", 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.579e-06 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953828+0000", 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42069 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960995+0000", 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025134091000000001, 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025404670000000001, 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42069 2024-09-16T22:16:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960995+0000", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960995+0000", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960996+0000", 2024-09-16T22:16:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960998+0000", 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961004+0000", 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5660000000000003e-06 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961011+0000", 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1189999999999999e-06 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42117 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968090+0000", 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018039320000000001, 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018362197, 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42117 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968090+0000", 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968090+0000", 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968090+0000", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968092+0000", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0889999999999998e-06 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968099+0000", 2024-09-16T22:16:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3679999999999998e-06 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968114+0000", 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5282e-05 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41678 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951326+0000", 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034803357, 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035174724999999997, 2024-09-16T22:16:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41678 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951326+0000", 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951326+0000", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951326+0000", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0999999999999999e-07 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951329+0000", 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8679999999999999e-06 2024-09-16T22:16:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951337+0000", 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0679999999999997e-06 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951343+0000", 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3800000000000002e-06 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41894 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952078+0000", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034051076999999999, 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034469122999999997, 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41894 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952078+0000", 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952078+0000", 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952079+0000", 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952081+0000", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952087+0000", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952092+0000", 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41958 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953835+0000", 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032294063999999997, 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032759414000000001, 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41958 2024-09-16T22:16:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953835+0000", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953835+0000", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953836+0000", 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:16:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953846+0000", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0132000000000001e-05 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953853+0000", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3679999999999996e-06 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953865+0000", 2024-09-16T22:16:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2204e-05 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42070 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.961018+0000", 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025110861000000002, 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025624472999999998, 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42070 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961018+0000", 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961018+0000", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961019+0000", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961021+0000", 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.08e-06 2024-09-16T22:16:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961028+0000", 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961033+0000", 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1869999999999996e-06 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42118 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968121+0000", 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018007850999999998, 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018569074000000001, 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42118 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968121+0000", 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968121+0000", 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968122+0000", 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1600000000000001e-07 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968124+0000", 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2330000000000001e-06 2024-09-16T22:16:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968130+0000", 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8289999999999996e-06 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968141+0000", 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1426999999999999e-05 2024-09-16T22:16:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41679 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951351+0000", 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034777925000000001, 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035387078000000002, 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41679 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951351+0000", 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951351+0000", 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951352+0000", 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6699999999999999e-07 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951354+0000", 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951361+0000", 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7610000000000002e-06 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951375+0000", 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3419e-05 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41895 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952414+0000", 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033714994999999998, 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034370922999999998, 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41895 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952414+0000", 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952414+0000", 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952415+0000", 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9400000000000005e-07 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952419+0000", 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7409999999999998e-06 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952430+0000", 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1456e-05 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952437+0000", 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7379999999999997e-06 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41959 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954027+0000", 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032102147999999997, 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032809577999999999, 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41959 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954027+0000", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954027+0000", 2024-09-16T22:16:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954028+0000", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7299999999999996e-07 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954031+0000", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5669999999999999e-06 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954042+0000", 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0754000000000001e-05 2024-09-16T22:16:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954049+0000", 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42071 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.961926+0000", 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024203295, 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024958556999999999, 2024-09-16T22:16:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42071 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961926+0000", 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961926+0000", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961927+0000", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3200000000000004e-07 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961938+0000", 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1369000000000001e-05 2024-09-16T22:16:49.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961951+0000", 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2977e-05 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961960+0000", 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9139999999999997e-06 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42119 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968149+0000", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017980066999999999, 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018789716000000001, 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42119 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968149+0000", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968149+0000", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968150+0000", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968152+0000", 2024-09-16T22:16:49.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0810000000000001e-06 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968158+0000", 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4899999999999997e-06 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968163+0000", 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8509999999999998e-06 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41680 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951383+0000", 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034746259000000002, 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035604087999999999, 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41680 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951383+0000", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951383+0000", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951383+0000", 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951386+0000", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951393+0000", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1579999999999999e-06 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951404+0000", 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0871e-05 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41896 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952492+0000", 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033637671000000001, 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034542849, 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41896 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952492+0000", 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952492+0000", 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952492+0000", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952495+0000", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952503+0000", 2024-09-16T22:16:49.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1389999999999995e-06 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952509+0000", 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7320000000000003e-06 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41960 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954060+0000", 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032069544999999998, 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033023834000000002, 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41960 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954060+0000", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954060+0000", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954060+0000", 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7599999999999997e-07 2024-09-16T22:16:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954063+0000", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5440000000000001e-06 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954070+0000", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0879999999999997e-06 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954075+0000", 2024-09-16T22:16:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2100000000000001e-06 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42072 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.961973+0000", 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024155795000000001, 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025157714000000001, 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42072 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961973+0000", 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961973+0000", 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961974+0000", 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.961987+0000", 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3066e-05 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962005+0000", 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7487e-05 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962012+0000", 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2289999999999997e-06 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42120 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968170+0000", 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017959486, 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019009278000000001, 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42120 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968170+0000", 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968170+0000", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968170+0000", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968178+0000", 2024-09-16T22:16:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4420000000000001e-06 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968184+0000", 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8719999999999996e-06 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968190+0000", 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2839999999999997e-06 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41681 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951419+0000", 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034710524999999999, 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035808939999999997, 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41681 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951419+0000", 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951419+0000", 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951419+0000", 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4300000000000003e-07 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951422+0000", 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9569999999999998e-06 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951430+0000", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3069999999999997e-06 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951435+0000", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2920000000000003e-06 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41897 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952518+0000", 2024-09-16T22:16:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033610876999999997, 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034744087999999999, 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41897 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952518+0000", 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952518+0000", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952519+0000", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952521+0000", 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0949999999999998e-06 2024-09-16T22:16:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952527+0000", 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1179999999999997e-06 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952532+0000", 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4569999999999998e-06 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41961 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954084+0000", 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032045215000000002, 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033201960000000003, 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41961 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954084+0000", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954084+0000", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954085+0000", 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954087+0000", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954094+0000", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954099+0000", 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4090000000000002e-06 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42073 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962036+0000", 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024093114999999998, 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025272685, 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42073 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962036+0000", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962036+0000", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962037+0000", 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-09-16T22:16:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962040+0000", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.818e-06 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962048+0000", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3259999999999996e-06 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962054+0000", 2024-09-16T22:16:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.105e-06 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42121 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968198+0000", 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017931472, 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019133504999999999, 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42121 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968198+0000", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968198+0000", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968198+0000", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2899999999999999e-07 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968200+0000", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968214+0000", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.34e-05 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968219+0000", 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1869999999999996e-06 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41682 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951443+0000", 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034685981999999997, 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035924973999999998, 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41682 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951443+0000", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951443+0000", 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951444+0000", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951446+0000", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951452+0000", 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.46e-06 2024-09-16T22:16:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951458+0000", 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3040000000000004e-06 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41898 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952539+0000", 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033590087999999997, 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034875452000000001, 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41898 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952539+0000", 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952539+0000", 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952540+0000", 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952542+0000", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1959999999999998e-06 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952548+0000", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952553+0000", 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9540000000000003e-06 2024-09-16T22:16:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41962 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954108+0000", 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032021257999999997, 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033354272999999997, 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41962 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954108+0000", 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954108+0000", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954108+0000", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954111+0000", 2024-09-16T22:16:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954117+0000", 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.545e-06 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954122+0000", 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8559999999999998e-06 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42074 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962071+0000", 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024058236, 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025439597000000001, 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42074 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962071+0000", 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962071+0000", 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962072+0000", 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000004e-07 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962074+0000", 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2419999999999999e-06 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962083+0000", 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4390000000000006e-06 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962097+0000", 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3851e-05 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42122 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968227+0000", 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017902162999999999, 2024-09-16T22:16:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019383524999999999, 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42122 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968227+0000", 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968227+0000", 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968228+0000", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.06e-07 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968230+0000", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2850000000000001e-06 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968237+0000", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3540000000000002e-06 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968242+0000", 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41685 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951465+0000", 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034664107, 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036198129000000002, 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41685 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951465+0000", 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951465+0000", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951466+0000", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951468+0000", 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.99e-06 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951473+0000", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5940000000000001e-06 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951477+0000", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1520000000000002e-06 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41899 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952559+0000", 2024-09-16T22:16:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033569922000000002, 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035151957999999997, 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41899 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952559+0000", 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952559+0000", 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952560+0000", 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.27e-07 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952562+0000", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8729999999999999e-06 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952567+0000", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952618+0000", 2024-09-16T22:16:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0336000000000001e-05 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41963 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954130+0000", 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031999475999999999, 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033632391999999997, 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41963 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954130+0000", 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954130+0000", 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954130+0000", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954132+0000", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954139+0000", 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6370000000000002e-06 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954144+0000", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42075 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962106+0000", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024023254000000001, 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025704049, 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42075 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962106+0000", 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962106+0000", 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962106+0000", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5600000000000001e-07 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962109+0000", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8310000000000001e-06 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962116+0000", 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6599999999999998e-06 2024-09-16T22:16:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962121+0000", 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5049999999999998e-06 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42123 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968249+0000", 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017880175000000002, 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019612592000000002, 2024-09-16T22:16:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42123 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968249+0000", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968249+0000", 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968249+0000", 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968252+0000", 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0899999999999999e-06 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968257+0000", 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9349999999999999e-06 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968262+0000", 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8740000000000003e-06 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41687 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951484+0000", 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034645687000000001, 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036426028999999999, 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41687 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951484+0000", 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951484+0000", 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951484+0000", 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951486+0000", 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8330000000000001e-06 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951491+0000", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489999999999996e-06 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951496+0000", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3819999999999997e-06 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41900 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952626+0000", 2024-09-16T22:16:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033503325, 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035338679999999997, 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41900 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952626+0000", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952626+0000", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952626+0000", 2024-09-16T22:16:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952633+0000", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0530000000000001e-06 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952641+0000", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952651+0000", 2024-09-16T22:16:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0023999999999999e-05 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41964 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954152+0000", 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031977482000000002, 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033859581999999999, 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41964 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954152+0000", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954152+0000", 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954152+0000", 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954154+0000", 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.097e-06 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954161+0000", 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3169999999999998e-06 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954166+0000", 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42076 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962128+0000", 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024000786, 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025911518000000001, 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42076 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962128+0000", 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962128+0000", 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962129+0000", 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962131+0000", 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0530000000000001e-06 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962139+0000", 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5379999999999997e-06 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962144+0000", 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42124 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968275+0000", 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017854667000000001, 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019788628999999999, 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42124 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968275+0000", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968275+0000", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968275+0000", 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:16:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968278+0000", 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968285+0000", 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9879999999999998e-06 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968290+0000", 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1869999999999996e-06 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41689 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951502+0000", 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034627321000000003, 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036586396, 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41689 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951502+0000", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951502+0000", 2024-09-16T22:16:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951502+0000", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951504+0000", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7400000000000001e-06 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951510+0000", 2024-09-16T22:16:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8359999999999997e-06 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951515+0000", 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8520000000000003e-06 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41901 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952658+0000", 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033471345, 2024-09-16T22:16:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035453459999999999, 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41901 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952658+0000", 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952658+0000", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952658+0000", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2200000000000004e-07 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952660+0000", 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-06 2024-09-16T22:16:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952666+0000", 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952671+0000", 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41965 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954173+0000", 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031956235999999999, 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033963211, 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41965 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954173+0000", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954173+0000", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954173+0000", 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:16:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954176+0000", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954182+0000", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320000000000004e-06 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954187+0000", 2024-09-16T22:16:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6770000000000003e-06 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42077 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962152+0000", 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023977722999999999, 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026007826000000001, 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42077 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962152+0000", 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962152+0000", 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962152+0000", 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962154+0000", 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962161+0000", 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5100000000000004e-06 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962166+0000", 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1789999999999999e-06 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42125 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968314+0000", 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017815583999999999, 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019893416000000001, 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42125 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968314+0000", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968314+0000", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968314+0000", 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-09-16T22:16:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968317+0000", 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968325+0000", 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-06 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968330+0000", 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41690 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951521+0000", 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034607909999999999, 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036733441999999998, 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41690 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951521+0000", 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951521+0000", 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951522+0000", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951524+0000", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8759999999999999e-06 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951530+0000", 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9939999999999997e-06 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951534+0000", 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1659999999999996e-06 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41902 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952677+0000", 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033452032999999999, 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035611413000000001, 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41902 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952677+0000", 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952677+0000", 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952678+0000", 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952679+0000", 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7799999999999999e-06 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952685+0000", 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0440000000000001e-06 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952690+0000", 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-06 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41966 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954376+0000", 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031753377999999999, 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033936538000000002, 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41966 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954376+0000", 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954376+0000", 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954377+0000", 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5600000000000005e-07 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954380+0000", 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2270000000000002e-06 2024-09-16T22:16:49.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954390+0000", 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8600000000000005e-06 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954396+0000", 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42078 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962173+0000", 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023956429000000001, 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026162274999999999, 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42078 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962173+0000", 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962173+0000", 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962173+0000", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962175+0000", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1289999999999999e-06 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962182+0000", 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1879999999999999e-06 2024-09-16T22:16:49.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962187+0000", 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0889999999999999e-06 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42126 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968339+0000", 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017790158, 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020022196999999999, 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42126 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968339+0000", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968339+0000", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968339+0000", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968342+0000", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968348+0000", 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1759999999999998e-06 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968362+0000", 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41691 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951540+0000", 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034589179999999997, 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036866474000000003, 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41691 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951540+0000", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951540+0000", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951540+0000", 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-16T22:16:49.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951549+0000", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949999999999994e-06 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951556+0000", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9419999999999998e-06 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951561+0000", 2024-09-16T22:16:49.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5070000000000002e-06 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41904 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952696+0000", 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033433619999999997, 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035762075999999997, 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41904 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952696+0000", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952696+0000", 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952696+0000", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952698+0000", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9089999999999998e-06 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952704+0000", 2024-09-16T22:16:49.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2280000000000003e-06 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952709+0000", 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4850000000000003e-06 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41968 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954407+0000", 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031722640000000003, 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034098468999999999, 2024-09-16T22:16:49.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41968 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954407+0000", 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954407+0000", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954407+0000", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954410+0000", 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.385e-06 2024-09-16T22:16:49.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954417+0000", 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954431+0000", 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3991999999999999e-05 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42079 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962193+0000", 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023936189, 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026368401999999999, 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42079 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962193+0000", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962193+0000", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962193+0000", 2024-09-16T22:16:49.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962195+0000", 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7090000000000001e-06 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962202+0000", 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5200000000000003e-06 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962207+0000", 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0459999999999996e-06 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42127 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968370+0000", 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017759079000000001, 2024-09-16T22:16:49.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020238542000000002, 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42127 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968370+0000", 2024-09-16T22:16:49.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968370+0000", 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968371+0000", 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968373+0000", 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968380+0000", 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7789999999999998e-06 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968385+0000", 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41692 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951569+0000", 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034560676999999998, 2024-09-16T22:16:49.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037091686999999998, 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41692 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951569+0000", 2024-09-16T22:16:49.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951569+0000", 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951569+0000", 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951571+0000", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4059999999999999e-06 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951578+0000", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1489999999999999e-06 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951588+0000", 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0601999999999999e-05 2024-09-16T22:16:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41906 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952715+0000", 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033414671, 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035996993999999997, 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41906 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952715+0000", 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952715+0000", 2024-09-16T22:16:49.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952715+0000", 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952722+0000", 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9360000000000002e-06 2024-09-16T22:16:49.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952728+0000", 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1979999999999998e-06 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952733+0000", 2024-09-16T22:16:49.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41970 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954441+0000", 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031688679999999997, 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034324904000000003, 2024-09-16T22:16:49.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41970 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954441+0000", 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954441+0000", 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954441+0000", 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954449+0000", 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4800000000000004e-06 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954456+0000", 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8800000000000008e-06 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954471+0000", 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4301000000000001e-05 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42080 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962228+0000", 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023900771000000001, 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026584304999999999, 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42080 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962228+0000", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962228+0000", 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962229+0000", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962231+0000", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7209999999999999e-06 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962237+0000", 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962242+0000", 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0549999999999999e-06 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42128 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968392+0000", 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017737258999999998, 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02047154, 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42128 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968392+0000", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968392+0000", 2024-09-16T22:16:49.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968392+0000", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968394+0000", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8870000000000001e-06 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968401+0000", 2024-09-16T22:16:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5420000000000002e-06 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968406+0000", 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41693 3.16 3.f497ce76 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951595+0000", 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034533917999999997, 2024-09-16T22:16:49.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037318939000000002, 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41693 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951595+0000", 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951595+0000", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951596+0000", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951598+0000", 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951604+0000", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.959e-06 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951608+0000", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41908 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952740+0000", 2024-09-16T22:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033388899, 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036231829, 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41908 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952740+0000", 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952740+0000", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952741+0000", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952743+0000", 2024-09-16T22:16:49.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952749+0000", 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3759999999999996e-06 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952754+0000", 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000003e-06 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.235 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41973 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.954480+0000", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.031649123000000001, 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034530159999999997, 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41973 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954480+0000", 2024-09-16T22:16:49.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.954480+0000", 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960389+0000", 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059087130000000003 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960402+0000", 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2942000000000001e-05 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960415+0000", 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2978e-05 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960423+0000", 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4770000000000003e-06 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42081 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962249+0000", 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023880025999999999, 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026865083000000001, 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42081 2024-09-16T22:16:49.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962249+0000", 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962249+0000", 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962250+0000", 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962252+0000", 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.159e-06 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962259+0000", 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3599999999999998e-06 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962264+0000", 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42129 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968413+0000", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017716366000000001, 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020730960999999999, 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42129 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968413+0000", 2024-09-16T22:16:49.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968413+0000", 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968413+0000", 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968415+0000", 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9690000000000001e-06 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968423+0000", 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7710000000000006e-06 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968428+0000", 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41874 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951615+0000", 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034514129999999997, 2024-09-16T22:16:49.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037573996999999998, 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41874 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951615+0000", 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951615+0000", 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951616+0000", 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951618+0000", 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0710000000000002e-06 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951624+0000", 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951628+0000", 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4939999999999997e-06 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41938 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952760+0000", 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033368916999999998, 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036483693999999997, 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41938 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952760+0000", 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952760+0000", 2024-09-16T22:16:49.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952761+0000", 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952763+0000", 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.119e-06 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952769+0000", 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0530000000000003e-06 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952773+0000", 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5789999999999997e-06 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42033 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960436+0000", 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025692999000000001, 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028856080999999999, 2024-09-16T22:16:49.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42033 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960436+0000", 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960436+0000", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960437+0000", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960439+0000", 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2400000000000002e-06 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960443+0000", 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0219999999999998e-06 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960446+0000", 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0230000000000001e-06 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42082 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962271+0000", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023858094999999999, 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027074516999999999, 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42082 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962271+0000", 2024-09-16T22:16:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962271+0000", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962272+0000", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962282+0000", 2024-09-16T22:16:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0813e-05 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962288+0000", 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8669999999999999e-06 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962293+0000", 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41875 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951634+0000", 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034494924000000003, 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037756975999999998, 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41875 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951634+0000", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951634+0000", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951635+0000", 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951637+0000", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9829999999999999e-06 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951643+0000", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951647+0000", 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3610000000000003e-06 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41939 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952780+0000", 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033349532000000001, 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036652933999999998, 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41939 2024-09-16T22:16:49.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952780+0000", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952780+0000", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952780+0000", 2024-09-16T22:16:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952782+0000", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8419999999999999e-06 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952787+0000", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3090000000000003e-06 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952791+0000", 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.121e-06 2024-09-16T22:16:49.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42035 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960450+0000", 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025679080999999999, 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029026922, 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42035 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960450+0000", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960450+0000", 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960450+0000", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6199999999999999e-07 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960452+0000", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4500000000000001e-06 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960455+0000", 2024-09-16T22:16:49.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1650000000000002e-06 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960459+0000", 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9230000000000004e-06 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42099 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962301+0000", 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023827872, 2024-09-16T22:16:49.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027231163999999999, 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42099 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962301+0000", 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962301+0000", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962302+0000", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962304+0000", 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4210000000000002e-06 2024-09-16T22:16:49.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962311+0000", 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320000000000004e-06 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962323+0000", 2024-09-16T22:16:49.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2500000000000001e-05 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41876 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951654+0000", 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034475351000000001, 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037926693999999997, 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41876 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951654+0000", 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951654+0000", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951654+0000", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951661+0000", 2024-09-16T22:16:49.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8190000000000003e-06 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951668+0000", 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6479999999999997e-06 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951673+0000", 2024-09-16T22:16:49.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7679999999999999e-06 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41940 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952798+0000", 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033331209000000001, 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036856367000000001, 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41940 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952798+0000", 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952798+0000", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952798+0000", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952800+0000", 2024-09-16T22:16:49.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952806+0000", 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7640000000000002e-06 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952817+0000", 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1001e-05 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42036 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960463+0000", 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02566622, 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029241646999999999, 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42036 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960463+0000", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960463+0000", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960463+0000", 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960465+0000", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.73e-06 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960471+0000", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7139999999999998e-06 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960476+0000", 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:16:49.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42100 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962331+0000", 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023798256, 2024-09-16T22:16:49.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027426623000000001, 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42100 2024-09-16T22:16:49.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962331+0000", 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962331+0000", 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962331+0000", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962338+0000", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3749999999999999e-06 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962343+0000", 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962355+0000", 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1868000000000001e-05 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41877 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951680+0000", 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034449683000000002, 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038125276999999999, 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41877 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951680+0000", 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951680+0000", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951680+0000", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951682+0000", 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9420000000000002e-06 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951688+0000", 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1279999999999996e-06 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951693+0000", 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4660000000000001e-06 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41941 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952824+0000", 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033305017999999999, 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037035846999999997, 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41941 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952824+0000", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952824+0000", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952825+0000", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952827+0000", 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952833+0000", 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1249999999999998e-06 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952838+0000", 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42037 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960483+0000", 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025646509000000001, 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029424913, 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42037 2024-09-16T22:16:49.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960483+0000", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960483+0000", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960483+0000", 2024-09-16T22:16:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960486+0000", 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960491+0000", 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.694e-06 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960496+0000", 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8659999999999997e-06 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42101 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962365+0000", 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023764199999999999, 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027607777999999999, 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42101 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962365+0000", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962365+0000", 2024-09-16T22:16:49.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962366+0000", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962368+0000", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5349999999999999e-06 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962374+0000", 2024-09-16T22:16:49.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6160000000000001e-06 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962379+0000", 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000001e-06 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41878 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951699+0000", 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034429842000000002, 2024-09-16T22:16:49.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038322266000000001, 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41878 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951699+0000", 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951699+0000", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951700+0000", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951702+0000", 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951708+0000", 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0669999999999997e-06 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951712+0000", 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2520000000000001e-06 2024-09-16T22:16:49.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41942 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952844+0000", 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033285125999999998, 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037223396999999998, 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41942 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952844+0000", 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952844+0000", 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952845+0000", 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952847+0000", 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0880000000000002e-06 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952853+0000", 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952857+0000", 2024-09-16T22:16:49.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.78e-06 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42038 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960547+0000", 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025581959000000001, 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029548657999999998, 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42038 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960547+0000", 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960547+0000", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960548+0000", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3300000000000002e-07 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960552+0000", 2024-09-16T22:16:49.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-06 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960562+0000", 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0732e-05 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960569+0000", 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6950000000000003e-06 2024-09-16T22:16:49.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42102 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962386+0000", 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023743651000000001, 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027733666000000001, 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42102 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962386+0000", 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962386+0000", 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962386+0000", 2024-09-16T22:16:49.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962388+0000", 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.978e-06 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962395+0000", 2024-09-16T22:16:49.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9240000000000001e-06 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962401+0000", 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0959999999999997e-06 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41879 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951719+0000", 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034410099, 2024-09-16T22:16:49.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038429821000000003, 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41879 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951719+0000", 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951719+0000", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951720+0000", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-07 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951722+0000", 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2120000000000002e-06 2024-09-16T22:16:49.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951728+0000", 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0510000000000002e-06 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951733+0000", 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0320000000000003e-06 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41943 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952864+0000", 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033264905999999997, 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037307954999999997, 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41943 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952864+0000", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952864+0000", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952865+0000", 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952867+0000", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1179999999999999e-06 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952873+0000", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0390000000000001e-06 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952878+0000", 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8300000000000003e-06 2024-09-16T22:16:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42039 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960579+0000", 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025550133999999999, 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029618398000000001, 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42039 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960579+0000", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960579+0000", 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960580+0000", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960582+0000", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960589+0000", 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2119999999999997e-06 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960594+0000", 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42103 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962409+0000", 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023720583, 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027811992000000001, 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42103 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962409+0000", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962409+0000", 2024-09-16T22:16:49.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962409+0000", 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962411+0000", 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962418+0000", 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962423+0000", 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9699999999999998e-06 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41880 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951739+0000", 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034389910000000003, 2024-09-16T22:16:49.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038505912000000003, 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41880 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951739+0000", 2024-09-16T22:16:49.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951739+0000", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951740+0000", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951742+0000", 2024-09-16T22:16:49.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7910000000000001e-06 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951747+0000", 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8980000000000001e-06 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951752+0000", 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4209999999999997e-06 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41944 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952884+0000", 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033245630999999998, 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037386258999999998, 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41944 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952884+0000", 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952884+0000", 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952884+0000", 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952890+0000", 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952897+0000", 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6220000000000003e-06 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952902+0000", 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8409999999999999e-06 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42040 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960601+0000", 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025528023, 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029715584999999999, 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42040 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960601+0000", 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960601+0000", 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960602+0000", 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960604+0000", 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9649999999999998e-06 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960610+0000", 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6950000000000003e-06 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960623+0000", 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2629999999999999e-05 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42104 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962430+0000", 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023699120000000001, 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027913805999999999, 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42104 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962430+0000", 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962430+0000", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962431+0000", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962433+0000", 2024-09-16T22:16:49.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.052e-06 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962439+0000", 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4670000000000001e-06 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962444+0000", 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8110000000000002e-06 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41881 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951758+0000", 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034371163000000003, 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038612023000000002, 2024-09-16T22:16:49.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41881 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951758+0000", 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951758+0000", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951758+0000", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951761+0000", 2024-09-16T22:16:49.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951767+0000", 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951779+0000", 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2041e-05 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41945 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952909+0000", 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033220144, 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037483722999999997, 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41945 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952909+0000", 2024-09-16T22:16:49.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952909+0000", 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952910+0000", 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:16:49.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952912+0000", 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.491e-06 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952923+0000", 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.119e-05 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952929+0000", 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4079999999999997e-06 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42041 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960631+0000", 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025498632, 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029789479000000001, 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42041 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960631+0000", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960631+0000", 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960631+0000", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960634+0000", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960641+0000", 2024-09-16T22:16:49.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.131e-06 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960646+0000", 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42105 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962451+0000", 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023678352, 2024-09-16T22:16:49.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027994377000000001, 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42105 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962451+0000", 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962451+0000", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962451+0000", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962454+0000", 2024-09-16T22:16:49.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2790000000000001e-06 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962460+0000", 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4760000000000003e-06 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962465+0000", 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41882 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951792+0000", 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034337286000000002, 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038696452999999999, 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41882 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951792+0000", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951792+0000", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951792+0000", 2024-09-16T22:16:49.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9299999999999998e-07 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951799+0000", 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9909999999999996e-06 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951807+0000", 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951819+0000", 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2585999999999999e-05 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41946 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952937+0000", 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033192167000000002, 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037578827000000002, 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41946 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952937+0000", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952937+0000", 2024-09-16T22:16:49.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952938+0000", 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952940+0000", 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952946+0000", 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6699999999999997e-06 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952951+0000", 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0189999999999997e-06 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42042 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960653+0000", 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025476447999999999, 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029885823999999998, 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42042 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960653+0000", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960653+0000", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960653+0000", 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3099999999999998e-07 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960655+0000", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9520000000000001e-06 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960662+0000", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8020000000000003e-06 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960667+0000", 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-06 2024-09-16T22:16:49.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42106 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962472+0000", 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023657508000000001, 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028091425999999999, 2024-09-16T22:16:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42106 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962472+0000", 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962472+0000", 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962472+0000", 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962474+0000", 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962481+0000", 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6590000000000001e-06 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962486+0000", 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.831e-06 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41883 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951827+0000", 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034301927000000003, 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038761684999999997, 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41883 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951827+0000", 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951827+0000", 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951828+0000", 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951830+0000", 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951837+0000", 2024-09-16T22:16:49.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951841+0000", 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41947 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952958+0000", 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.033170843999999998, 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037656028000000001, 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41947 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952958+0000", 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952958+0000", 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952959+0000", 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-16T22:16:49.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952970+0000", 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1001e-05 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952977+0000", 2024-09-16T22:16:49.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1130000000000003e-06 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952982+0000", 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.045e-06 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42043 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960674+0000", 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025454971, 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029963564000000002, 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42043 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960674+0000", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960674+0000", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960675+0000", 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960677+0000", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960683+0000", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8549999999999998e-06 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960688+0000", 2024-09-16T22:16:49.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4229999999999996e-06 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42107 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962492+0000", 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023637063999999999, 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028185137999999998, 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42107 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962492+0000", 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962492+0000", 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962493+0000", 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962495+0000", 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2620000000000001e-06 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962502+0000", 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5499999999999997e-06 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962508+0000", 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.392 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41884 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951848+0000", 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034280892, 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038854629000000002, 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41884 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951848+0000", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951848+0000", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951849+0000", 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-16T22:16:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951851+0000", 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.889e-06 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951856+0000", 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7259999999999999e-06 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951861+0000", 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2030000000000002e-06 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41948 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953378+0000", 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032750885, 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037350118000000002, 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41948 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953378+0000", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953378+0000", 2024-09-16T22:16:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953379+0000", 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.06e-07 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953383+0000", 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6480000000000001e-06 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953394+0000", 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953400+0000", 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999998e-06 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42044 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960763+0000", 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025365859000000001, 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029990243, 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42044 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960763+0000", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960763+0000", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960764+0000", 2024-09-16T22:16:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3e-07 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960778+0000", 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3495999999999999e-05 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960787+0000", 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3400000000000004e-06 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960793+0000", 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7130000000000002e-06 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42108 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962515+0000", 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023614349999999999, 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028264088999999999, 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42108 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962515+0000", 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962515+0000", 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962515+0000", 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962517+0000", 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962524+0000", 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0380000000000002e-06 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962537+0000", 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2332e-05 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41885 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951867+0000", 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034262697000000002, 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038935640000000001, 2024-09-16T22:16:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41885 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951867+0000", 2024-09-16T22:16:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951867+0000", 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951867+0000", 2024-09-16T22:16:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951869+0000", 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951875+0000", 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6919999999999999e-06 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951879+0000", 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6229999999999996e-06 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41949 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953413+0000", 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032716701000000001, 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037414795000000001, 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41949 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953413+0000", 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953413+0000", 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953413+0000", 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953416+0000", 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.638e-06 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953423+0000", 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953437+0000", 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3963e-05 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42045 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960802+0000", 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025327615000000001, 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0300548, 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42045 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960802+0000", 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960802+0000", 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960802+0000", 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960804+0000", 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1050000000000002e-06 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960811+0000", 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960816+0000", 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0130000000000001e-06 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42109 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962545+0000", 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023584619000000001, 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028353630000000001, 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42109 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962545+0000", 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962545+0000", 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962545+0000", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962554+0000", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4549999999999995e-06 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962562+0000", 2024-09-16T22:16:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9810000000000003e-06 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962567+0000", 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1390000000000003e-06 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41886 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951886+0000", 2024-09-16T22:16:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034242938000000001, 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039045600999999999, 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41886 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951886+0000", 2024-09-16T22:16:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951886+0000", 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951887+0000", 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951889+0000", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9970000000000001e-06 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951894+0000", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4410000000000001e-06 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951898+0000", 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2699999999999998e-06 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41950 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953446+0000", 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032683265000000003, 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037511398000000001, 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41950 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953446+0000", 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953446+0000", 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953446+0000", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953449+0000", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4090000000000001e-06 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953456+0000", 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8379999999999996e-06 2024-09-16T22:16:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953467+0000", 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1396000000000001e-05 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42046 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960823+0000", 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025305897000000001, 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03017123, 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42046 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960823+0000", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960823+0000", 2024-09-16T22:16:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960824+0000", 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960826+0000", 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960832+0000", 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960837+0000", 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8930000000000004e-06 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42110 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962574+0000", 2024-09-16T22:16:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023554792000000001, 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028466736999999999, 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42110 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962574+0000", 2024-09-16T22:16:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962574+0000", 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962575+0000", 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962577+0000", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999998e-06 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962583+0000", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.156e-06 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962588+0000", 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8949999999999997e-06 2024-09-16T22:16:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41887 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951905+0000", 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034224638000000002, 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039193457000000001, 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41887 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951905+0000", 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951905+0000", 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951905+0000", 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951907+0000", 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-09-16T22:16:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951913+0000", 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7810000000000002e-06 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951922+0000", 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2769999999999994e-06 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41951 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953475+0000", 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032654034999999998, 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037670518, 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41951 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953475+0000", 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953475+0000", 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953476+0000", 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953478+0000", 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.328e-06 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953487+0000", 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5399999999999996e-06 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953492+0000", 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2410000000000003e-06 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42047 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960844+0000", 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025284838, 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030355271999999999, 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42047 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960844+0000", 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960844+0000", 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960845+0000", 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2300000000000002e-07 2024-09-16T22:16:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960847+0000", 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9850000000000001e-06 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960853+0000", 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960858+0000", 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42111 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962595+0000", 2024-09-16T22:16:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023534204999999999, 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028652469999999999, 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42111 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962595+0000", 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962595+0000", 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962595+0000", 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962597+0000", 2024-09-16T22:16:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8920000000000001e-06 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962604+0000", 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4980000000000003e-06 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962611+0000", 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999998e-06 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41888 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951929+0000", 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034199905000000003, 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039368918000000003, 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41888 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951929+0000", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951929+0000", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951930+0000", 2024-09-16T22:16:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951937+0000", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9199999999999998e-06 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951943+0000", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7179999999999999e-06 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951948+0000", 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-09-16T22:16:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41952 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953500+0000", 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032629241000000003, 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037845286999999998, 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41952 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953500+0000", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953500+0000", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953500+0000", 2024-09-16T22:16:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953503+0000", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.379e-06 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953510+0000", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2699999999999999e-06 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953515+0000", 2024-09-16T22:16:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42048 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960866+0000", 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025263724000000001, 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030533225000000001, 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42048 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960866+0000", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960866+0000", 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960866+0000", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960868+0000", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960874+0000", 2024-09-16T22:16:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960879+0000", 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000001e-06 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42112 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.962619+0000", 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023510161000000002, 2024-09-16T22:16:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028827545999999999, 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42112 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962619+0000", 2024-09-16T22:16:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962619+0000", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967944+0000", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053248770000000004 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967955+0000", 2024-09-16T22:16:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0835e-05 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967969+0000", 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3725e-05 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967977+0000", 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8650000000000007e-06 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41889 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951956+0000", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034173658000000003, 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039558062999999997, 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41889 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951956+0000", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951956+0000", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951956+0000", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7400000000000003e-07 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951958+0000", 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0779999999999998e-06 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951965+0000", 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1809999999999998e-06 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951969+0000", 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5360000000000003e-06 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41953 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953523+0000", 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032606127999999998, 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038039101999999998, 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41953 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953523+0000", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953523+0000", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953524+0000", 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953526+0000", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953532+0000", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953537+0000", 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7509999999999999e-06 2024-09-16T22:16:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42049 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960888+0000", 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02524084, 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030722330999999999, 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42049 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960888+0000", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960888+0000", 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960889+0000", 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960891+0000", 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.204e-06 2024-09-16T22:16:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960897+0000", 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7799999999999997e-06 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960910+0000", 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3523000000000001e-05 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42113 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.967994+0000", 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018134852, 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023640890000000001, 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42113 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967994+0000", 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967994+0000", 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967995+0000", 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.967998+0000", 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5450000000000002e-06 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968005+0000", 2024-09-16T22:16:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6299999999999998e-06 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968011+0000", 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9359999999999996e-06 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41890 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.951976+0000", 2024-09-16T22:16:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034153519, 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039684527999999997, 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41890 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951976+0000", 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951976+0000", 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951976+0000", 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951978+0000", 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951984+0000", 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5860000000000004e-06 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.951997+0000", 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3117e-05 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41954 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032402893000000002, 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037971963999999997, 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41954 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953727+0000", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9100000000000003e-07 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953731+0000", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6090000000000001e-06 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953742+0000", 2024-09-16T22:16:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1351e-05 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953749+0000", 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1520000000000003e-06 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42050 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960923+0000", 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025206457000000002, 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030799874000000001, 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42050 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960923+0000", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960923+0000", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960923+0000", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960926+0000", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-06 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960933+0000", 2024-09-16T22:16:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2699999999999999e-06 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960944+0000", 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1355999999999999e-05 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42114 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968019+0000", 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018109791, 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023912449999999998, 2024-09-16T22:16:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42114 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968019+0000", 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968019+0000", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968020+0000", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968022+0000", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.204e-06 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968032+0000", 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968038+0000", 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.066e-06 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41891 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.952005+0000", 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.034124133000000001, 2024-09-16T22:16:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039977094999999997, 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41891 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952005+0000", 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952005+0000", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952006+0000", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952008+0000", 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952015+0000", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6730000000000003e-06 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.952021+0000", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41955 3.e 3.2881fe6e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.953760+0000", 2024-09-16T22:16:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.032369013000000002, 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038288857000000003, 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41955 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953760+0000", 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953760+0000", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953761+0000", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3799999999999997e-07 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953763+0000", 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-16T22:16:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953771+0000", 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1459999999999999e-06 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.953776+0000", 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4650000000000002e-06 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42067 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.960952+0000", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02517701, 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031146419000000002, 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42067 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960952+0000", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960952+0000", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960953+0000", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960955+0000", 2024-09-16T22:16:49.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2790000000000001e-06 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960962+0000", 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6120000000000004e-06 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.960967+0000", 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0170000000000004e-06 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42115 3.e 3.1d1dc6ae (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.968046+0000", 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018083116, 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024119246, 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42115 2024-09-16T22:16:49.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968046+0000", 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968046+0000", 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968047+0000", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5999999999999999e-07 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968049+0000", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.39e-06 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968055+0000", 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3539999999999996e-06 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.968061+0000", 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2850000000000002e-06 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 142 2024-09-16T22:16:49.507 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:49.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:16:49.509 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:49.510+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:49.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:49.512 INFO:teuthology.orchestra.run.smithi031.stdout:2311 punch from 0x246bc9b to 0x2476adb, (0xae40 bytes) 2024-09-16T22:16:49.512 INFO:teuthology.orchestra.run.smithi031.stdout:2312 write 0x1a45a81 thru 0x1a4986d (0x3ded bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2313 read 0x1e5d50d thru 0x1e69497 (0xbf8b bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2317 write 0x7f8a685 thru 0x7f8fbe5 (0x5561 bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2318 writesame 0xaa133ef thru 0xaa19879 data_size 0x1(0x648b bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2320 write 0x6300288 thru 0x630b97c (0xb6f5 bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2322 write 0x45a651 thru 0x467394 (0xcd44 bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2324 writesame 0xc86d7ce thru 0xc87bdb6 data_size 0x17(0xe5e9 bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2325 write 0xc0ee78b thru 0xc0f1404 (0x2c7a bytes) 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.513 INFO:teuthology.orchestra.run.smithi031.stdout:2328 read 0x21ec0bd thru 0x21fb87d (0xf7c1 bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2330 write 0x95ed60e thru 0x95f5214 (0x7c07 bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2331 punch from 0x2c90b2b to 0x2c9b741, (0xac16 bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2332 write 0x9856f95 thru 0x985b936 (0x49a2 bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2333 read 0x4fba2e9 thru 0x4fc3c62 (0x997a bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2338 write 0x976b226 thru 0x976d63c (0x2417 bytes) 2024-09-16T22:16:49.514 INFO:teuthology.orchestra.run.smithi031.stdout:2339 trunc from 0xc87bdb7 to 0x97ad80d 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2343 write 0x31a8795 thru 0x31b7421 (0xec8d bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2344 punch from 0x3ac5ccc to 0x3acb65a, (0x598e bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2345 punch from 0x83c669 to 0x848f87, (0xc91e bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2346 punch from 0x1a9a508 to 0x1a9dabb, (0x35b3 bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2349 trunc from 0x97ad80d to 0x916738d 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2351 trunc from 0x916738d to 0x9aa77a1 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2352 punch from 0x507041a to 0x50725ea, (0x21d0 bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2353 read 0x1de5dad thru 0x1df2985 (0xcbd9 bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2354 punch from 0x531178f to 0x531cdbf, (0xb630 bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2356 read 0x7cfecf4 thru 0x7d06a16 (0x7d23 bytes) 2024-09-16T22:16:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:2357 trunc from 0x9aa77a1 to 0x80c1389 2024-09-16T22:16:49.516 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T22:16:49.516 INFO:teuthology.orchestra.run.smithi031.stdout:2358 write 0x839b735 thru 0x83a7c0d (0xc4d9 bytes) 2024-09-16T22:16:49.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2359 trunc from 0x83a7c0e to 0xa13a1ed 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2361 write 0xc867d62 thru 0xc877596 (0xf835 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2363 writesame 0x8c22fdc thru 0x8c30880 data_size 0x15(0xd8a5 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2365 writesame 0x9b03cda thru 0x9b0e40c data_size 0x1(0xa733 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2367 read 0xb5e6f5d thru 0xb5efc6e (0x8d12 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2369 write 0x5da34da thru 0x5da7dcd (0x48f4 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2370 punch from 0x9bdf631 to 0x9bef567, (0xff36 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2371 writesame 0x51038c4 thru 0x51110bf data_size 0x2e(0xd7fc bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2372 read 0x1ce0289 thru 0x1ce1be5 (0x195d bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2374 write 0xb7903a6 thru 0xb79b327 (0xaf82 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2375 writesame 0xed9ce8c thru 0xeda2bdc data_size 0x1(0x5d51 bytes) 2024-09-16T22:16:49.521 INFO:teuthology.orchestra.run.smithi031.stdout:2376 trunc from 0xeda2bdd to 0xdda61ac 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2377 write 0x36b03a9 thru 0x36b26cd (0x2325 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2379 writesame 0xa18e380 thru 0xa19adba data_size 0x1(0xca3b bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2381 write 0x851ca43 thru 0x8526896 (0x9e54 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2383 writesame 0x9ff0a32 thru 0x9fffd25 data_size 0x49(0xf2f4 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2385 trunc from 0xdda61ac to 0xe354b33 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2386 writesame 0x4062dab thru 0x4071d9a data_size 0x58(0xeff0 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2388 write 0x77f78ac thru 0x7800aa7 (0x91fc bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2391 read 0xb7fb748 thru 0xb801490 (0x5d49 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2392 write 0xbec5951 thru 0xbecb592 (0x5c42 bytes) 2024-09-16T22:16:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:2393 clone 22 order 20 su 524288 sc 11 2024-09-16T22:16:49.610 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:49.611+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:49.661 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:49.662 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:49.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:49.712 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:49.714+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:49.811 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:49.813+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41329 3.16 3:6adc9550:::rbd_data.11cd43ed039f.0000000000000013:16 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.910498+0000", 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.91287425700000002, 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048078405999999997, 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41329 2024-09-16T22:16:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910498+0000", 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910498+0000", 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910498+0000", 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6e-07 2024-09-16T22:16:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910499+0000", 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910502+0000", 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2399999999999999e-06 2024-09-16T22:16:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.910505+0000", 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3649999999999998e-06 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.956048+0000", 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045543637999999997 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.956126+0000", 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7842000000000005e-05 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.958576+0000", 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002449943 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41468 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 120832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915305+0000", 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90806726999999998, 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047577090000000002, 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41468 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915305+0000", 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915305+0000", 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915305+0000", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915316+0000", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0746e-05 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915323+0000", 2024-09-16T22:16:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915333+0000", 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0176999999999999e-05 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962712+0000", 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047378324999999999 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962796+0000", 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3955999999999994e-05 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962882+0000", 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5927999999999995e-05 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41469 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 129024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915346+0000", 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90802602499999996, 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047739942, 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41469 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915346+0000", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915346+0000", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915346+0000", 2024-09-16T22:16:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8699999999999995e-07 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915361+0000", 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4977000000000001e-05 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915370+0000", 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2959999999999999e-06 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915375+0000", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6879999999999996e-06 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.962935+0000", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047559509999999999 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963009+0000", 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4337000000000003e-05 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963086+0000", 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6647e-05 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41474 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 2048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915384+0000", 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90798765400000003, 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047878500999999997, 2024-09-16T22:16:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41474 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915384+0000", 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915384+0000", 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915385+0000", 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6599999999999996e-07 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915394+0000", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8920000000000006e-06 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915402+0000", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3459999999999994e-06 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915408+0000", 2024-09-16T22:16:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7930000000000003e-06 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963113+0000", 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047705368999999997 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963181+0000", 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7673000000000001e-05 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963263+0000", 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1861999999999997e-05 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41475 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 10240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915416+0000", 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90795606500000003, 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048015348999999999, 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41475 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915416+0000", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915416+0000", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915416+0000", 2024-09-16T22:16:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915418+0000", 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915425+0000", 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5289999999999997e-06 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915430+0000", 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4280000000000004e-06 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963289+0000", 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047859187999999997 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963351+0000", 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1983999999999995e-05 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963431+0000", 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9709999999999994e-05 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41476 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 18432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915437+0000", 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90793462599999997, 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048157697999999999, 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41476 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915437+0000", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915437+0000", 2024-09-16T22:16:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915438+0000", 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915440+0000", 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915446+0000", 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915451+0000", 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963458+0000", 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048006785000000003 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963516+0000", 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7438999999999997e-05 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963595+0000", 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9257000000000004e-05 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41477 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 26624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915458+0000", 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90791367700000003, 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048308614999999999, 2024-09-16T22:16:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41477 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915458+0000", 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915458+0000", 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915459+0000", 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915460+0000", 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8109999999999999e-06 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915467+0000", 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5679999999999996e-06 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915472+0000", 2024-09-16T22:16:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963621+0000", 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048149441000000001 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963681+0000", 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9193000000000001e-05 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963767+0000", 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6204000000000003e-05 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41478 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915478+0000", 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90789350000000002, 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048464097999999997, 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41478 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915478+0000", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915478+0000", 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915479+0000", 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915481+0000", 2024-09-16T22:16:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.88e-06 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915487+0000", 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5239999999999997e-06 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915492+0000", 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963797+0000", 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048304813000000002 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963855+0000", 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7797000000000002e-05 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963942+0000", 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7707999999999994e-05 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41479 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915499+0000", 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.907873288, 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048609860999999997, 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41479 2024-09-16T22:16:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915499+0000", 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915499+0000", 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915499+0000", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915501+0000", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.779e-06 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915507+0000", 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915512+0000", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.963967+0000", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048455148000000003 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964029+0000", 2024-09-16T22:16:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1914000000000004e-05 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964108+0000", 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9324000000000001e-05 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41480 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 51200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915518+0000", 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90785371100000001, 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048742115000000003, 2024-09-16T22:16:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41480 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915518+0000", 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915518+0000", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915519+0000", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915520+0000", 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7290000000000001e-06 2024-09-16T22:16:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915527+0000", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6880000000000002e-06 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915532+0000", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964133+0000", 2024-09-16T22:16:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048601236999999999 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964194+0000", 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1074999999999994e-05 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964260+0000", 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6074000000000005e-05 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41481 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 59392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.915538+0000", 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.90783347199999997, 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048873735000000001, 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41481 2024-09-16T22:16:49.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915538+0000", 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915538+0000", 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915539+0000", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915541+0000", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9209999999999999e-06 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915548+0000", 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.878e-06 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.915560+0000", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.216e-05 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964285+0000", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048725355999999997 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964348+0000", 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3230000000000003e-05 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.964412+0000", 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3789999999999997e-05 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41520 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.923862+0000", 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.89951010399999998, 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047519638000000003, 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41520 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923862+0000", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923862+0000", 2024-09-16T22:16:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923862+0000", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923864+0000", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.826e-06 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923870+0000", 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7660000000000004e-06 2024-09-16T22:16:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923874+0000", 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1200000000000004e-06 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971232+0000", 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047358443 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971305+0000", 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2263999999999998e-05 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971381+0000", 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6755999999999995e-05 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41521 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.923880+0000", 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.89949227499999995, 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047723663, 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41521 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923880+0000", 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923880+0000", 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923880+0000", 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923882+0000", 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.804e-06 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923888+0000", 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0870000000000003e-06 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923892+0000", 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0940000000000001e-06 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971437+0000", 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047544788999999997 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971523+0000", 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6483999999999994e-05 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971603+0000", 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0044999999999997e-05 2024-09-16T22:16:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41554 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 133120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.923897+0000", 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.89947443000000005, 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047898282, 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41554 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923897+0000", 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923897+0000", 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923898+0000", 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.27e-07 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923900+0000", 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7069999999999999e-06 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923905+0000", 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.604e-06 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923909+0000", 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999997e-06 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971636+0000", 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047726600000000001 2024-09-16T22:16:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971716+0000", 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0402999999999996e-05 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971796+0000", 2024-09-16T22:16:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9351000000000006e-05 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41555 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 141312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.923915+0000", 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.89945688400000001, 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048080155999999999, 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41555 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923915+0000", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.923915+0000", 2024-09-16T22:16:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.930461+0000", 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065460229999999998 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.930467+0000", 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6339999999999998e-06 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.930483+0000", 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6365999999999999e-05 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.930491+0000", 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9699999999999999e-06 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971828+0000", 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.041336699999999997 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971905+0000", 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7544000000000002e-05 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.971995+0000", 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9919000000000004e-05 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41655 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.937034+0000", 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88633803600000005, 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047496957999999999, 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41655 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937034+0000", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937034+0000", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937034+0000", 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8699999999999995e-07 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937037+0000", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4930000000000001e-06 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937043+0000", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3520000000000003e-06 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937050+0000", 2024-09-16T22:16:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4160000000000001e-06 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984282+0000", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047232279000000002 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984384+0000", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101915 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984531+0000", 2024-09-16T22:16:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147016 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41656 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.937056+0000", 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88631538700000001, 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047743440999999998, 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41656 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937056+0000", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.937056+0000", 2024-09-16T22:16:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938548+0000", 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001491806 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938554+0000", 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8560000000000003e-06 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938567+0000", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2508e-05 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938575+0000", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.089e-06 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984566+0000", 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045991162000000002 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984672+0000", 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010581699999999999 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.984800+0000", 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012820299999999999 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41671 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.938772+0000", 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88460012899999996, 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047435669, 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41671 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938772+0000", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938772+0000", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938772+0000", 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938774+0000", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9350000000000001e-06 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938780+0000", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938785+0000", 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7770000000000002e-06 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986044+0000", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047258733999999997 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986127+0000", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3201999999999994e-05 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986207+0000", 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0696000000000001e-05 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41672 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.938791+0000", 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88458127099999995, 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047639003999999999, 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41672 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938791+0000", 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938791+0000", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938791+0000", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1200000000000003e-07 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938793+0000", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.796e-06 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938799+0000", 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000001e-06 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938803+0000", 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1479999999999999e-06 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986239+0000", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047436461999999999 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986318+0000", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9073999999999995e-05 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986430+0000", 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111369 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:41673 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.938808+0000", 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88456393499999997, 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047860648999999998, 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:49.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 41673 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938808+0000", 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.938808+0000", 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.948946+0000", 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010138374 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.948954+0000", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2810000000000003e-06 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.948965+0000", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.948973+0000", 2024-09-16T22:16:49.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986489+0000", 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037516655000000003 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986571+0000", 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1181999999999996e-05 2024-09-16T22:16:49.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.986669+0000", 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7969999999999999e-05 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:49.923 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:49.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:49.926 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:49.926+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:50.025 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:50.027+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:50.072 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:50.073 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:50.073 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:50.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:50.126 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:50.128+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:50.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:50.237 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:50.239+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:16:50.308 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T22:16:50.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T22:16:50.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:50.328 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:50.329+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.927086758000002, 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31471 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857960077, 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020335977000000002, 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31471 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496624+0000", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8589999999999999e-06 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496642+0000", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142e-05 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496648+0000", 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516812+0000", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020163871 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516873+0000", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0956999999999999e-05 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516953+0000", 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9983999999999999e-05 2024-09-16T22:16:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31473 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857919788, 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020442202, 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31473 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496658+0000", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496661+0000", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-06 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496668+0000", 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3259999999999998e-06 2024-09-16T22:16:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496673+0000", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4040000000000003e-06 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516979+0000", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020305882000000001 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517041+0000", 2024-09-16T22:16:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1307000000000001e-05 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517099+0000", 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8817000000000003e-05 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31475 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857897132, 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021013756000000001, 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31475 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496682+0000", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496689+0000", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.652e-06 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496694+0000", 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-16T22:16:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517567+0000", 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020872472 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517631+0000", 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3998000000000006e-05 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517694+0000", 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3106000000000002e-05 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31567 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857815196000001, 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02048005, 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31567 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496765+0000", 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.219e-06 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496772+0000", 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496777+0000", 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8790000000000002e-06 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517122+0000", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020345268999999999 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517182+0000", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0195000000000002e-05 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517242+0000", 2024-09-16T22:16:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9512999999999998e-05 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31568 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857793693, 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020607819999999999, 2024-09-16T22:16:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31568 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496784+0000", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496786+0000", 2024-09-16T22:16:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496801+0000", 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5517000000000001e-05 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496807+0000", 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7060000000000001e-06 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517265+0000", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020458418999999999 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517329+0000", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3967999999999995e-05 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517391+0000", 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1914000000000004e-05 2024-09-16T22:16:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31569 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857762107999999, 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020725204000000001, 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31569 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496818+0000", 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.464e-06 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496826+0000", 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8159999999999994e-06 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496831+0000", 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489999999999996e-06 2024-09-16T22:16:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517414+0000", 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020582975999999999 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517480+0000", 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5260999999999997e-05 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517540+0000", 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0537e-05 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31575 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857737896, 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021677216999999999, 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31575 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496840+0000", 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496842+0000", 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5110000000000002e-06 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496849+0000", 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496854+0000", 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517723+0000", 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020868653000000001 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517796+0000", 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399000000000007e-05 2024-09-16T22:16:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518516+0000", 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072027500000000004 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31576 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857715908999999, 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021185559999999999, 2024-09-16T22:16:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31576 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496862+0000", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496864+0000", 2024-09-16T22:16:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3549999999999999e-06 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496870+0000", 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119999999999998e-06 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496875+0000", 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517895+0000", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021019867000000001 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517972+0000", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6711000000000005e-05 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518047+0000", 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.483e-05 2024-09-16T22:16:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31585 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857695168999999, 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021312514000000001, 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31585 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496884+0000", 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496890+0000", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1829999999999999e-06 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496896+0000", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0660000000000003e-06 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518071+0000", 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021175678 2024-09-16T22:16:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518134+0000", 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3263999999999996e-05 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518194+0000", 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9871999999999999e-05 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31586 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857674199, 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021438353, 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31586 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496912+0000", 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5879999999999998e-06 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496925+0000", 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3648999999999999e-05 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496931+0000", 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518219+0000", 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021288517 2024-09-16T22:16:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518279+0000", 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0145000000000001e-05 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518341+0000", 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1717000000000006e-05 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31593 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857637767, 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021611253, 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31593 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496940+0000", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496943+0000", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1109999999999999e-06 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496949+0000", 2024-09-16T22:16:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4699999999999999e-06 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496954+0000", 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000001e-06 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518371+0000", 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021416918 2024-09-16T22:16:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518442+0000", 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1662000000000006e-05 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518550+0000", 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108034 2024-09-16T22:16:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31594 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857616382, 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021755911999999999, 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31594 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-16T22:16:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496963+0000", 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496969+0000", 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496975+0000", 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.288e-06 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518576+0000", 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021600891000000001 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518640+0000", 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3905000000000006e-05 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518717+0000", 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31595 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857595061, 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021907319000000001, 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31595 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496985+0000", 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8849999999999999e-06 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496991+0000", 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2759999999999997e-06 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496997+0000", 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518748+0000", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021751047999999999 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518811+0000", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518889+0000", 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8641999999999997e-05 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31603 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857573791, 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022076842999999999, 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31603 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497004+0000", 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497006+0000", 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0549999999999998e-06 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497011+0000", 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5860000000000004e-06 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497016+0000", 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4800000000000003e-06 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518950+0000", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021934694000000001 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519017+0000", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6538000000000006e-05 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519080+0000", 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31604 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857555169999999, 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022218702, 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31604 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497025+0000", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3259999999999998e-06 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497031+0000", 2024-09-16T22:16:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8370000000000002e-06 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497035+0000", 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519107+0000", 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022072692000000001 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519172+0000", 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4672999999999996e-05 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519241+0000", 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8367000000000005e-05 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31605 3.11 3.538ddd91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:50.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857535662, 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022265193999999999, 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31605 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497042+0000", 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:50.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497052+0000", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0073e-05 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497059+0000", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.007e-06 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497064+0000", 2024-09-16T22:16:50.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6559999999999997e-06 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519299+0000", 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022234296000000001 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519307+0000", 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7500000000000003e-06 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31627 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857269241000001, 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021960872999999999, 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31627 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497310+0000", 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9980000000000002e-06 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497316+0000", 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497321+0000", 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6909999999999996e-06 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519262+0000", 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021940867999999999 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519269+0000", 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6590000000000001e-06 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31628 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.857227587000001, 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021947498999999999, 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31628 2024-09-16T22:16:50.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497350+0000", 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497352+0000", 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4679999999999999e-06 2024-09-16T22:16:50.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497359+0000", 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497365+0000", 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.327e-06 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519290+0000", 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021925153999999999 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519297+0000", 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1489999999999997e-06 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:42589 3.16 e51/23 3:6bec85e1:::rbd_data.11cd49ca855d.0000000000000003:head v 51'165, mlcod=51'162)", 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.1887977519999999, 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028158426, 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:50.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165781+0000", 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7910000000000001e-06 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169904+0000", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041225469999999998 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169940+0000", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6180000000000003e-05 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169955+0000", 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5707000000000001e-05 2024-09-16T22:16:50.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.170045+0000", 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0002999999999999e-05 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.170076+0000", 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0037999999999999e-05 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:50.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.193881+0000", 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023805702000000002 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.193938+0000", 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6458000000000001e-05 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:50.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:50.456 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:50.457 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:50.459+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:50.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:50.459 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone21 from 0xe354b33 (overlap 0x2f326e6) to 0x1c51756 2024-09-16T22:16:50.503 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T22:16:50.558 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:50.559+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:50.602 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:50.602 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:50.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45453 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586277+0000", 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011836943000000001, 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011858628, 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45453 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586277+0000", 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586277+0000", 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586277+0000", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.44e-07 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586279+0000", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.827e-06 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586285+0000", 2024-09-16T22:16:50.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9089999999999997e-06 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586289+0000", 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4050000000000002e-06 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45629 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596517+0000", 2024-09-16T22:16:50.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001596613, 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016747019999999999, 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45629 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596517+0000", 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596517+0000", 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596517+0000", 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596520+0000", 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596526+0000", 2024-09-16T22:16:50.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4280000000000001e-06 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596531+0000", 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1490000000000002e-06 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45454 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586296+0000", 2024-09-16T22:16:50.610 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011817461, 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011941221, 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45454 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586296+0000", 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586296+0000", 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586297+0000", 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6899999999999998e-07 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586298+0000", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.728e-06 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586304+0000", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8459999999999996e-06 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586314+0000", 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3999999999999998e-06 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45630 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596539+0000", 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001574909, 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017436369999999999, 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45630 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596539+0000", 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596539+0000", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596539+0000", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2300000000000002e-07 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596548+0000", 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9379999999999998e-06 2024-09-16T22:16:50.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596556+0000", 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3110000000000006e-06 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596562+0000", 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2569999999999998e-06 2024-09-16T22:16:50.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45456 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586320+0000", 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.01179359, 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012006339, 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45456 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586320+0000", 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586320+0000", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586320+0000", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586322+0000", 2024-09-16T22:16:50.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.866e-06 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586328+0000", 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9179999999999999e-06 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586332+0000", 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3229999999999999e-06 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45631 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596570+0000", 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015440079999999999, 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001804431, 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45631 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596570+0000", 2024-09-16T22:16:50.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596570+0000", 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596570+0000", 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-07 2024-09-16T22:16:50.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596572+0000", 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2689999999999998e-06 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596579+0000", 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:16:50.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596583+0000", 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5070000000000002e-06 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45458 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586338+0000", 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011775333000000001, 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012082089000000001, 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45458 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586338+0000", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586338+0000", 2024-09-16T22:16:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586339+0000", 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.77e-07 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586340+0000", 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.72e-06 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586346+0000", 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3809999999999998e-06 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586351+0000", 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2619999999999998e-06 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45632 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596590+0000", 2024-09-16T22:16:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001523939, 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018746139999999999, 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45632 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596590+0000", 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596590+0000", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596590+0000", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596592+0000", 2024-09-16T22:16:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.001e-06 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596598+0000", 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9050000000000002e-06 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596603+0000", 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3990000000000003e-06 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45460 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586357+0000", 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011756681999999999, 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012148856, 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45460 2024-09-16T22:16:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586357+0000", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586357+0000", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586357+0000", 2024-09-16T22:16:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.27e-07 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586359+0000", 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.86e-06 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586365+0000", 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586369+0000", 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4689999999999999e-06 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45633 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596609+0000", 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0015043400000000001, 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001945018, 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45633 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596609+0000", 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596609+0000", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596610+0000", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596611+0000", 2024-09-16T22:16:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.683e-06 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596617+0000", 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7270000000000004e-06 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596622+0000", 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6999999999999999e-06 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45462 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586375+0000", 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011738446, 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012222533000000001, 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45462 2024-09-16T22:16:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586375+0000", 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586375+0000", 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586376+0000", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586377+0000", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586383+0000", 2024-09-16T22:16:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586388+0000", 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4270000000000001e-06 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45634 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596628+0000", 2024-09-16T22:16:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0014858860000000001, 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020128860000000002, 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45634 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596628+0000", 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596628+0000", 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596628+0000", 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:16:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596630+0000", 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8250000000000001e-06 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596636+0000", 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7060000000000001e-06 2024-09-16T22:16:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596640+0000", 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5950000000000001e-06 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45464 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586393+0000", 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011720526, 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012291514, 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45464 2024-09-16T22:16:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586393+0000", 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586393+0000", 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586393+0000", 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586395+0000", 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.745e-06 2024-09-16T22:16:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586401+0000", 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0569999999999998e-06 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586406+0000", 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4850000000000003e-06 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45635 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596646+0000", 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001467341, 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020803869999999999, 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45635 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596646+0000", 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596646+0000", 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596647+0000", 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.15e-07 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596648+0000", 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7409999999999999e-06 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596654+0000", 2024-09-16T22:16:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2650000000000004e-06 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596658+0000", 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5589999999999999e-06 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45467 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586411+0000", 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011702064999999999, 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012360779000000001, 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45467 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586411+0000", 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586411+0000", 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586412+0000", 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586414+0000", 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8810000000000001e-06 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586420+0000", 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9159999999999998e-06 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586424+0000", 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9249999999999997e-06 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45636 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596665+0000", 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001449047, 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021521679999999999, 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45636 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596665+0000", 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596665+0000", 2024-09-16T22:16:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596665+0000", 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.08e-07 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596667+0000", 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9520000000000001e-06 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596673+0000", 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9549999999999997e-06 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596680+0000", 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45470 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586430+0000", 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011683826, 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012440279, 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45470 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586430+0000", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586430+0000", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586430+0000", 2024-09-16T22:16:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7400000000000003e-07 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586437+0000", 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3040000000000001e-06 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586443+0000", 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259999999999997e-06 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586447+0000", 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8930000000000004e-06 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45637 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596687+0000", 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001426683, 2024-09-16T22:16:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002232843, 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45637 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596687+0000", 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596687+0000", 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596687+0000", 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596689+0000", 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-06 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596697+0000", 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8439999999999998e-06 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596706+0000", 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4009999999999997e-06 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45473 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586454+0000", 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011659722000000001, 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012512572, 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45473 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586454+0000", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586454+0000", 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586454+0000", 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0999999999999998e-07 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586456+0000", 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.88e-06 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586462+0000", 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7030000000000003e-06 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586466+0000", 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45638 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596712+0000", 2024-09-16T22:16:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001401259, 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023006099999999998, 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45638 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596712+0000", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596712+0000", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596713+0000", 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596715+0000", 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596721+0000", 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0730000000000001e-06 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596726+0000", 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0039999999999999e-06 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45475 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586472+0000", 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011641347999999999, 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012586346, 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45475 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586472+0000", 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586472+0000", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586473+0000", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9900000000000001e-07 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586474+0000", 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.761e-06 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586480+0000", 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7130000000000002e-06 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586489+0000", 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3000000000000007e-06 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45639 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596732+0000", 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0013814509999999999, 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023724459999999998, 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45639 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596732+0000", 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596732+0000", 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596732+0000", 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596734+0000", 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9989999999999998e-06 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596741+0000", 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0769999999999996e-06 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596747+0000", 2024-09-16T22:16:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1059999999999996e-06 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45477 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586496+0000", 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011617808, 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012657139, 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45477 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586496+0000", 2024-09-16T22:16:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586496+0000", 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586496+0000", 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2099999999999997e-07 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586501+0000", 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586508+0000", 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1009999999999997e-06 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586519+0000", 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1703e-05 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45640 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596754+0000", 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0013591440000000001, 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024456790000000001, 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45640 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596754+0000", 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596754+0000", 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596755+0000", 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596757+0000", 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596763+0000", 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2009999999999996e-06 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596769+0000", 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0410000000000003e-06 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45433 3.7 3:e6e85fec:::rbd_header.11cdbd6a305e:head [watch reconnect cookie 140027814257024 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585809+0000", 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012304532, 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013442323000000001, 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45433 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585809+0000", 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585809+0000", 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585809+0000", 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585812+0000", 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1280000000000002e-06 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585817+0000", 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8309999999999997e-06 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585822+0000", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.34e-06 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.598851+0000", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013029682000000001 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.598894+0000", 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2246000000000002e-05 2024-09-16T22:16:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45479 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586526+0000", 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011587637, 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012780257, 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45479 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586526+0000", 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586526+0000", 2024-09-16T22:16:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586526+0000", 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586532+0000", 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8220000000000003e-06 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586539+0000", 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586544+0000", 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45641 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596805+0000", 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0013085589999999999, 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025470950000000001, 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45641 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596805+0000", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596805+0000", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596806+0000", 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0800000000000004e-07 2024-09-16T22:16:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596808+0000", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596815+0000", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.46e-06 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596820+0000", 2024-09-16T22:16:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2480000000000004e-06 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45434 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 4096~2048 out=2048b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585828+0000", 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.01228567, 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013575475, 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45434 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585828+0000", 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585828+0000", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585834+0000", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1040000000000003e-06 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585836+0000", 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.418e-06 2024-09-16T22:16:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585844+0000", 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585849+0000", 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4159999999999997e-06 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.599188+0000", 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013339437000000001 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.599305+0000", 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116662 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45562 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593038+0000", 2024-09-16T22:16:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00507524, 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068462849999999997, 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45562 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593038+0000", 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593038+0000", 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593039+0000", 2024-09-16T22:16:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6799999999999999e-07 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593044+0000", 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593054+0000", 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5070000000000006e-06 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593061+0000", 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45658 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596835+0000", 2024-09-16T22:16:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001278946, 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031006369999999998, 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45658 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596835+0000", 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596835+0000", 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596835+0000", 2024-09-16T22:16:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6600000000000002e-07 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596845+0000", 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596853+0000", 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8590000000000005e-06 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596859+0000", 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8730000000000004e-06 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45563 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593073+0000", 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0050400749999999998, 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069119949999999998, 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45563 2024-09-16T22:16:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593073+0000", 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593073+0000", 2024-09-16T22:16:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593074+0000", 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7599999999999997e-07 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593077+0000", 2024-09-16T22:16:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6790000000000001e-06 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593085+0000", 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0460000000000006e-06 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593090+0000", 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4689999999999996e-06 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45659 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596867+0000", 2024-09-16T22:16:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0012468539999999999, 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031639210000000001, 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45659 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596867+0000", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596867+0000", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596867+0000", 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596869+0000", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0669999999999999e-06 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596876+0000", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0219999999999999e-06 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596881+0000", 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9660000000000004e-06 2024-09-16T22:16:50.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45564 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593097+0000", 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0050161299999999997, 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069820710000000003, 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45564 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593097+0000", 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593097+0000", 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593098+0000", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593100+0000", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1040000000000001e-06 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593108+0000", 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0199999999999994e-06 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593113+0000", 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0189999999999997e-06 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45660 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596888+0000", 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001225753, 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032368090000000002, 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45660 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596888+0000", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596888+0000", 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596888+0000", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596890+0000", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596896+0000", 2024-09-16T22:16:50.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596901+0000", 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5750000000000002e-06 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45565 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593120+0000", 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0049930829999999997, 2024-09-16T22:16:50.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070883689999999997, 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45565 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593120+0000", 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593120+0000", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593121+0000", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593123+0000", 2024-09-16T22:16:50.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1619999999999998e-06 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593130+0000", 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7129999999999999e-06 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593135+0000", 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.293e-06 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45661 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596909+0000", 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001204866, 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033478190000000001, 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45661 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596909+0000", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596909+0000", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596909+0000", 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2899999999999999e-07 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596911+0000", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596917+0000", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.28e-06 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596922+0000", 2024-09-16T22:16:50.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1450000000000001e-06 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45438 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 36864~2048 out=2048b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585916+0000", 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012197141999999999, 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014394437, 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45438 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585916+0000", 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585916+0000", 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585917+0000", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585919+0000", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8199999999999999e-06 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585924+0000", 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6180000000000003e-06 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585929+0000", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3150000000000002e-06 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600137+0000", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014208292000000001 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600228+0000", 2024-09-16T22:16:50.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0921000000000006e-05 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45566 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593142+0000", 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0049719880000000001, 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072258660000000001, 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45566 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593142+0000", 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593142+0000", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593142+0000", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593144+0000", 2024-09-16T22:16:50.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.041e-06 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593150+0000", 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1500000000000004e-06 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593155+0000", 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9830000000000004e-06 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45662 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596927+0000", 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0011862039999999999, 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034887550000000001, 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45662 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596927+0000", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596927+0000", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596928+0000", 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6100000000000002e-07 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596930+0000", 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8729999999999999e-06 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596936+0000", 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0549999999999996e-06 2024-09-16T22:16:50.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596940+0000", 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45439 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 45056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585935+0000", 2024-09-16T22:16:50.755 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012179038, 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014532774, 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45439 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.756 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585935+0000", 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585935+0000", 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585935+0000", 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585943+0000", 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2600000000000005e-06 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585949+0000", 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9900000000000002e-06 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585954+0000", 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3210000000000004e-06 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600366+0000", 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014411775999999999 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600449+0000", 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3153999999999997e-05 2024-09-16T22:16:50.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45567 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593162+0000", 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0049518130000000002, 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073615729999999997, 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45567 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593162+0000", 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593162+0000", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593162+0000", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593164+0000", 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9999999999999999e-06 2024-09-16T22:16:50.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593171+0000", 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3810000000000004e-06 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593176+0000", 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9570000000000001e-06 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45663 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596946+0000", 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0011679279999999999, 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003624052, 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45663 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596946+0000", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596946+0000", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596946+0000", 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:16:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596948+0000", 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6810000000000001e-06 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596953+0000", 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6910000000000002e-06 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596958+0000", 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5650000000000003e-06 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45440 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 53248~2048 out=2048b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585961+0000", 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012152876, 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01466581, 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:50.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45440 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585961+0000", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585961+0000", 2024-09-16T22:16:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585961+0000", 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585968+0000", 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7889999999999997e-06 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585975+0000", 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5410000000000003e-06 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585980+0000", 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7430000000000002e-06 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600557+0000", 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014576565999999999 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600593+0000", 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6619999999999998e-05 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45568 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593181+0000", 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0049321149999999999, 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075015999999999998, 2024-09-16T22:16:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45568 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593181+0000", 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593181+0000", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593182+0000", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593184+0000", 2024-09-16T22:16:50.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.21e-06 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593190+0000", 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2990000000000002e-06 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593198+0000", 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3300000000000001e-06 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45664 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596986+0000", 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001127685, 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037460829999999999, 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45664 2024-09-16T22:16:50.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596986+0000", 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596986+0000", 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596986+0000", 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596989+0000", 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4830000000000002e-06 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596996+0000", 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999999e-06 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597002+0000", 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8250000000000001e-06 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45441 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.585986+0000", 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012127305, 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014796198999999999, 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45441 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585986+0000", 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585986+0000", 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585987+0000", 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5600000000000001e-07 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585989+0000", 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8989999999999999e-06 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585995+0000", 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9200000000000001e-06 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.585999+0000", 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600749+0000", 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014749788999999999 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45569 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593205+0000", 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0049083579999999998, 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007637057, 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45569 2024-09-16T22:16:50.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593205+0000", 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593205+0000", 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593206+0000", 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2399999999999998e-07 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593208+0000", 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593214+0000", 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4119999999999998e-06 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593220+0000", 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.215e-06 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45665 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597012+0000", 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001101472, 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038754610000000002, 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45665 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597012+0000", 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597012+0000", 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597013+0000", 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999995e-07 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597015+0000", 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0049999999999999e-06 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597021+0000", 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.511e-06 2024-09-16T22:16:50.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597026+0000", 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0459999999999996e-06 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45442 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586005+0000", 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012108686, 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014931933, 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45442 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586005+0000", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586005+0000", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586005+0000", 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586007+0000", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8109999999999999e-06 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586013+0000", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586017+0000", 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.600914+0000", 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014896639999999999 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45570 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593226+0000", 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0048873830000000004, 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077626209999999999, 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45570 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593226+0000", 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593226+0000", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593227+0000", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593229+0000", 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:16:50.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593235+0000", 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1750000000000002e-06 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593240+0000", 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9490000000000004e-06 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45666 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597033+0000", 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001080831, 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004004461, 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45666 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597033+0000", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597033+0000", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597033+0000", 2024-09-16T22:16:50.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.03e-07 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597035+0000", 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597041+0000", 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5500000000000002e-06 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597045+0000", 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6280000000000004e-06 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45443 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 77824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586037+0000", 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012076320999999999, 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015050902999999999, 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45443 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586037+0000", 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586037+0000", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586038+0000", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586040+0000", 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586046+0000", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586051+0000", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7180000000000004e-06 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601053+0000", 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015002557999999999 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45571 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593246+0000", 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0048675960000000001, 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078968040000000003, 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45571 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593246+0000", 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593246+0000", 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593246+0000", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593248+0000", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0169999999999999e-06 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593255+0000", 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1340000000000001e-06 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593259+0000", 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.729e-06 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45667 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597051+0000", 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0010625890000000001, 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041373099999999999, 2024-09-16T22:16:50.810 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45667 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597051+0000", 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597051+0000", 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597051+0000", 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3999999999999997e-07 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597053+0000", 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.778e-06 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597061+0000", 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0250000000000002e-06 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597073+0000", 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1688000000000001e-05 2024-09-16T22:16:50.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45444 3.7 3:e61bd652:::rbd_data.11cd43ed039f.000000000000005a:16 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586058+0000", 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012055872, 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015184649, 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45444 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586058+0000", 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586058+0000", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586058+0000", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586060+0000", 2024-09-16T22:16:50.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.801e-06 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586066+0000", 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5439999999999998e-06 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586071+0000", 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0950000000000003e-06 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601220+0000", 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015149564000000001 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45572 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593265+0000", 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0048483650000000003, 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080269720000000003, 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45572 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593265+0000", 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593265+0000", 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593266+0000", 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7599999999999998e-07 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593267+0000", 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8989999999999999e-06 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593274+0000", 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593283+0000", 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7159999999999998e-06 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45668 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597079+0000", 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001034662, 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042611089999999999, 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45668 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597079+0000", 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597079+0000", 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597079+0000", 2024-09-16T22:16:50.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4299999999999999e-07 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597081+0000", 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9250000000000002e-06 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597087+0000", 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0959999999999997e-06 2024-09-16T22:16:50.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597093+0000", 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0159999999999997e-06 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45445 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586077+0000", 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012036784999999999, 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01529173, 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45445 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586077+0000", 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586077+0000", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586077+0000", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586079+0000", 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8810000000000001e-06 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586085+0000", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7350000000000001e-06 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586095+0000", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6330000000000008e-06 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601361+0000", 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015266617 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45573 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593291+0000", 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0048226299999999996, 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081046669999999994, 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45573 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593291+0000", 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593291+0000", 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593291+0000", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7399999999999998e-07 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593298+0000", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5239999999999997e-06 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593305+0000", 2024-09-16T22:16:50.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0539999999999997e-06 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593310+0000", 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1919999999999996e-06 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45669 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597099+0000", 2024-09-16T22:16:50.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0010145880000000001, 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043297140000000001, 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45669 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597099+0000", 2024-09-16T22:16:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597099+0000", 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597099+0000", 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597101+0000", 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.683e-06 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597108+0000", 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2529999999999998e-06 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597113+0000", 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7029999999999997e-06 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45446 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586100+0000", 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012013098999999999, 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01535335, 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45446 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586100+0000", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586100+0000", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586101+0000", 2024-09-16T22:16:50.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9500000000000003e-07 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586103+0000", 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8810000000000001e-06 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586108+0000", 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3140000000000003e-06 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586117+0000", 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9439999999999994e-06 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45574 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593317+0000", 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0047965689999999997, 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081611419999999997, 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45574 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593317+0000", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593317+0000", 2024-09-16T22:16:50.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593317+0000", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593319+0000", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593326+0000", 2024-09-16T22:16:50.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3239999999999999e-06 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593331+0000", 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.989e-06 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45670 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597125+0000", 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00098843900000000003, 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043834420000000004, 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45670 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597125+0000", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597125+0000", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597126+0000", 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-16T22:16:50.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597128+0000", 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597134+0000", 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8009999999999998e-06 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597139+0000", 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.831e-06 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45447 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586123+0000", 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011990143, 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015428826, 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45447 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586123+0000", 2024-09-16T22:16:50.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586123+0000", 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586124+0000", 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:16:50.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586126+0000", 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8309999999999999e-06 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586131+0000", 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3909999999999997e-06 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586136+0000", 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6909999999999996e-06 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45575 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593337+0000", 2024-09-16T22:16:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0047764419999999997, 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.008260992, 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45575 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593337+0000", 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593337+0000", 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593337+0000", 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593339+0000", 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593348+0000", 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6649999999999992e-06 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593353+0000", 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0379999999999999e-06 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45671 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597164+0000", 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00094945100000000003, 2024-09-16T22:16:50.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044831469999999998, 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45671 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.858 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597164+0000", 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597164+0000", 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597165+0000", 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597167+0000", 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5299999999999999e-06 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597175+0000", 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2989999999999999e-06 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597180+0000", 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45448 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586142+0000", 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011971614, 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015550794, 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45448 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586142+0000", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586142+0000", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586142+0000", 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586144+0000", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8390000000000001e-06 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586150+0000", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6779999999999997e-06 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586154+0000", 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3170000000000003e-06 2024-09-16T22:16:50.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45576 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593360+0000", 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0047532620000000003, 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084527430000000004, 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45576 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593360+0000", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593360+0000", 2024-09-16T22:16:50.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593361+0000", 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-07 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593363+0000", 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0530000000000001e-06 2024-09-16T22:16:50.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593369+0000", 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0599999999999996e-06 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593375+0000", 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45672 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597188+0000", 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00092600600000000005, 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004678298, 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45672 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597188+0000", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597188+0000", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597188+0000", 2024-09-16T22:16:50.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597190+0000", 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.95e-06 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597196+0000", 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597201+0000", 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9250000000000003e-06 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45449 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586160+0000", 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011953706, 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015753949, 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45449 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586160+0000", 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586160+0000", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586160+0000", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586162+0000", 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586168+0000", 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586172+0000", 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5650000000000003e-06 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.875 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45577 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.593382+0000", 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0047316779999999996, 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00857809, 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45577 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593382+0000", 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593382+0000", 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593382+0000", 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9599999999999999e-07 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593384+0000", 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0269999999999998e-06 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593391+0000", 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1070000000000001e-06 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.593395+0000", 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5839999999999996e-06 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45673 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.597207+0000", 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00090620100000000003, 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048013309999999998, 2024-09-16T22:16:50.879 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45673 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597207+0000", 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597207+0000", 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597208+0000", 2024-09-16T22:16:50.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597210+0000", 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.852e-06 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597216+0000", 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.597221+0000", 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8489999999999997e-06 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45450 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586178+0000", 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011935226, 2024-09-16T22:16:50.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015877321999999999, 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45450 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586178+0000", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586178+0000", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586179+0000", 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:16:50.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586185+0000", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1890000000000004e-06 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586191+0000", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586196+0000", 2024-09-16T22:16:50.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45626 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596434+0000", 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001679743, 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056702059999999997, 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45626 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596434+0000", 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596434+0000", 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596435+0000", 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.09e-07 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596439+0000", 2024-09-16T22:16:50.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2930000000000002e-06 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596451+0000", 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1885999999999999e-05 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596459+0000", 2024-09-16T22:16:50.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7379999999999994e-06 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45451 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586201+0000", 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011912139, 2024-09-16T22:16:50.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015948173, 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45451 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586201+0000", 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586201+0000", 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586202+0000", 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586204+0000", 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9089999999999998e-06 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586209+0000", 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4580000000000001e-06 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586214+0000", 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5399999999999997e-06 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45627 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596472+0000", 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.001641946, 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057237570000000003, 2024-09-16T22:16:50.896 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45627 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596472+0000", 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596472+0000", 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596472+0000", 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596475+0000", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.554e-06 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596482+0000", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3540000000000002e-06 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596488+0000", 2024-09-16T22:16:50.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5659999999999997e-06 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45786 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.601358+0000", 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9967551, 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088352799999999996, 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.900 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45786 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601358+0000", 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601358+0000", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601359+0000", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5300000000000003e-07 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601363+0000", 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1010000000000002e-06 2024-09-16T22:16:50.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601373+0000", 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2860000000000005e-06 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601380+0000", 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2010000000000002e-06 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45452 3.7 3.4a6bd867 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.586252+0000", 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011861903, 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016038420000000001, 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45452 2024-09-16T22:16:50.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586252+0000", 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586252+0000", 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586252+0000", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999999e-07 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586258+0000", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5790000000000003e-06 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586265+0000", 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1080000000000004e-06 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.586270+0000", 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0579999999999997e-06 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45628 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.596496+0000", 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0016173019999999999, 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005839782, 2024-09-16T22:16:50.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45628 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596496+0000", 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596496+0000", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596497+0000", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596499+0000", 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-16T22:16:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596505+0000", 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1580000000000002e-06 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.596510+0000", 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7539999999999997e-06 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.910 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:45787 3.7 3.cd09a9e7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.601392+0000", 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4294967295.9967213, 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099197299999999994, 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45787 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601392+0000", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601392+0000", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601393+0000", 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601395+0000", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5639999999999999e-06 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601402+0000", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.365e-06 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.601407+0000", 2024-09-16T22:16:50.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0499999999999999e-06 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 80 2024-09-16T22:16:50.914 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:50.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:50.918 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:50.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:50.919 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:50.917+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:50.920 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-16T22:16:50.575739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326349879,"num_objects":1345,"num_object_clones":81,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":992,"num_read":47838,"num_read_kb":173396,"num_write":6452,"num_write_kb":131687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6381,"ondisk_log_size":6381,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":362720,"kb_used_data":83812,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754612000,"statfs":{"total":773094113280,"available":772722688000,"internally_reserved":0,"allocated":85823488,"data_stored":95351024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11041751,"num_objects":419,"num_object_clones":15,"num_object_copies":838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":419,"num_whiteouts":368,"num_read":6154,"num_read_kb":29361,"num_write":1378,"num_write_kb":7592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":144719,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000668"},"pg_stats":[{"pgid":"3.1f","version":"49'224","reported_seq":1751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.566998+0000","last_change":"2024-09-16T22:16:17.136383+0000","last_active":"2024-09-16T22:16:46.566998+0000","last_peered":"2024-09-16T22:16:46.566998+0000","last_clean":"2024-09-16T22:16:46.566998+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:16:46.566998+0000","last_undegraded":"2024-09-16T22:16:46.566998+0000","last_fullsized":"2024-09-16T22:16:46.566998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:37.254425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095346500000000002,"stat_sum":{"num_bytes":11999249,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":34,"num_read":2502,"num_read_kb":5938,"num_write":194,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"47'108","reported_seq":512,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294429+0000","last_change":"2024-09-16T22:16:17.136432+0000","last_active":"2024-09-16T22:16:46.294429+0000","last_peered":"2024-09-16T22:16:46.294429+0000","last_clean":"2024-09-16T22:16:46.294429+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:16:46.294429+0000","last_undegraded":"2024-09-16T22:16:46.294429+0000","last_fullsized":"2024-09-16T22:16:46.294429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:04.096142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078321800000000002,"stat_sum":{"num_bytes":7084633,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":26,"num_read":566,"num_read_kb":12322,"num_write":84,"num_write_kb":30982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"49'146","reported_seq":829,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.568620+0000","last_change":"2024-09-16T22:16:17.128822+0000","last_active":"2024-09-16T22:16:46.568620+0000","last_peered":"2024-09-16T22:16:46.568620+0000","last_clean":"2024-09-16T22:16:46.568620+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:16:46.568620+0000","last_undegraded":"2024-09-16T22:16:46.568620+0000","last_fullsized":"2024-09-16T22:16:46.568620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:04.455146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019949680000000002,"stat_sum":{"num_bytes":6518353,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":19,"num_read":1088,"num_read_kb":1983,"num_write":125,"num_write_kb":982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"48'131","reported_seq":1225,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.467202+0000","last_change":"2024-09-16T22:16:17.131219+0000","last_active":"2024-09-16T22:16:46.467202+0000","last_peered":"2024-09-16T22:16:46.467202+0000","last_clean":"2024-09-16T22:16:46.467202+0000","last_became_active":"2024-09-16T22:16:15.135556+0000","last_became_peered":"2024-09-16T22:16:15.135556+0000","last_unstale":"2024-09-16T22:16:46.467202+0000","last_undegraded":"2024-09-16T22:16:46.467202+0000","last_fullsized":"2024-09-16T22:16:46.467202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:26:53.306823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052703799999999996,"stat_sum":{"num_bytes":4523322,"num_objects":44,"num_object_clones":3,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":33,"num_read":1119,"num_read_kb":4057,"num_write":99,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"49'212","reported_seq":903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.528025+0000","last_change":"2024-09-16T22:16:17.263803+0000","last_active":"2024-09-16T22:16:46.528025+0000","last_peered":"2024-09-16T22:16:46.528025+0000","last_clean":"2024-09-16T22:16:46.528025+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:16:46.528025+0000","last_undegraded":"2024-09-16T22:16:46.528025+0000","last_fullsized":"2024-09-16T22:16:46.528025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:31.454818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053437600000000004,"stat_sum":{"num_bytes":8276329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":25,"num_read":1710,"num_read_kb":1644,"num_write":183,"num_write_kb":1779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"49'164","reported_seq":5619,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.602300+0000","last_change":"2024-09-16T22:16:17.131812+0000","last_active":"2024-09-16T22:16:46.602300+0000","last_peered":"2024-09-16T22:16:46.602300+0000","last_clean":"2024-09-16T22:16:46.602300+0000","last_became_active":"2024-09-16T22:16:15.126160+0000","last_became_peered":"2024-09-16T22:16:15.126160+0000","last_unstale":"2024-09-16T22:16:46.602300+0000","last_undegraded":"2024-09-16T22:16:46.602300+0000","last_fullsized":"2024-09-16T22:16:46.602300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:52.480013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010623589999999999,"stat_sum":{"num_bytes":15445043,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":5525,"num_read_kb":5598,"num_write":131,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"48'99","reported_seq":384,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.425255+0000","last_change":"2024-09-16T22:16:17.263495+0000","last_active":"2024-09-16T22:16:46.425255+0000","last_peered":"2024-09-16T22:16:46.425255+0000","last_clean":"2024-09-16T22:16:46.425255+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:16:46.425255+0000","last_undegraded":"2024-09-16T22:16:46.425255+0000","last_fullsized":"2024-09-16T22:16:46.425255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:22.625830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040510500000000001,"stat_sum":{"num_bytes":9877981,"num_objects":39,"num_object_clones":2,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":27,"num_read":257,"num_read_kb":898,"num_write":69,"num_write_kb":422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"49'210","reported_seq":314,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294526+0000","last_change":"2024-09-16T22:16:17.136712+0000","last_active":"2024-09-16T22:16:46.294526+0000","last_peered":"2024-09-16T22:16:46.294526+0000","last_clean":"2024-09-16T22:16:46.294526+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:16:46.294526+0000","last_undegraded":"2024-09-16T22:16:46.294526+0000","last_fullsized":"2024-09-16T22:16:46.294526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:52.131127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096381500000000003,"stat_sum":{"num_bytes":4589609,"num_objects":37,"num_object_clones":1,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":31,"num_read":592,"num_read_kb":616,"num_write":252,"num_write_kb":563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"49'388","reported_seq":1079,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294610+0000","last_change":"2024-09-16T22:16:17.135852+0000","last_active":"2024-09-16T22:16:46.294610+0000","last_peered":"2024-09-16T22:16:46.294610+0000","last_clean":"2024-09-16T22:16:46.294610+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:16:46.294610+0000","last_undegraded":"2024-09-16T22:16:46.294610+0000","last_fullsized":"2024-09-16T22:16:46.294610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:30.009725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042473899999999999,"stat_sum":{"num_bytes":15147821,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":29,"num_read":2047,"num_read_kb":8519,"num_write":564,"num_write_kb":17525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"47'137","reported_seq":1407,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276133+0000","last_change":"2024-09-16T22:16:17.127257+0000","last_active":"2024-09-16T22:16:46.276133+0000","last_peered":"2024-09-16T22:16:46.276133+0000","last_clean":"2024-09-16T22:16:46.276133+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:16:46.276133+0000","last_undegraded":"2024-09-16T22:16:46.276133+0000","last_fullsized":"2024-09-16T22:16:46.276133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:13.884897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093962400000000001,"stat_sum":{"num_bytes":13638927,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":1231,"num_read_kb":6032,"num_write":111,"num_write_kb":2031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"49'401","reported_seq":796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418090+0000","last_change":"2024-09-16T22:16:17.135778+0000","last_active":"2024-09-16T22:16:46.418090+0000","last_peered":"2024-09-16T22:16:46.418090+0000","last_clean":"2024-09-16T22:16:46.418090+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:16:46.418090+0000","last_undegraded":"2024-09-16T22:16:46.418090+0000","last_fullsized":"2024-09-16T22:16:46.418090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:36.889739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041336299999999998,"stat_sum":{"num_bytes":12518884,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":1765,"num_read_kb":7416,"num_write":558,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"49'144","reported_seq":448,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560653+0000","last_change":"2024-09-16T22:16:17.128475+0000","last_active":"2024-09-16T22:16:46.560653+0000","last_peered":"2024-09-16T22:16:46.560653+0000","last_clean":"2024-09-16T22:16:46.560653+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:16:46.560653+0000","last_undegraded":"2024-09-16T22:16:46.560653+0000","last_fullsized":"2024-09-16T22:16:46.560653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:42.401536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001932436,"stat_sum":{"num_bytes":9306351,"num_objects":36,"num_object_clones":1,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":28,"num_read":391,"num_read_kb":1711,"num_write":135,"num_write_kb":531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275996+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:46.275996+0000","last_peered":"2024-09-16T22:16:46.275996+0000","last_clean":"2024-09-16T22:16:46.275996+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:46.275996+0000","last_undegraded":"2024-09-16T22:16:46.275996+0000","last_fullsized":"2024-09-16T22:16:46.275996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"49'422","reported_seq":6951,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.482335+0000","last_change":"2024-09-16T22:16:17.263952+0000","last_active":"2024-09-16T22:16:46.482335+0000","last_peered":"2024-09-16T22:16:46.482335+0000","last_clean":"2024-09-16T22:16:46.482335+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:16:46.482335+0000","last_undegraded":"2024-09-16T22:16:46.482335+0000","last_fullsized":"2024-09-16T22:16:46.482335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:53.349620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063812899999999997,"stat_sum":{"num_bytes":12184410,"num_objects":44,"num_object_clones":2,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":8813,"num_read_kb":13552,"num_write":532,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280213+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:46.280213+0000","last_peered":"2024-09-16T22:16:46.280213+0000","last_clean":"2024-09-16T22:16:46.280213+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:46.280213+0000","last_undegraded":"2024-09-16T22:16:46.280213+0000","last_fullsized":"2024-09-16T22:16:46.280213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"49'145","reported_seq":633,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.474629+0000","last_change":"2024-09-16T22:16:17.264006+0000","last_active":"2024-09-16T22:16:46.474629+0000","last_peered":"2024-09-16T22:16:46.474629+0000","last_clean":"2024-09-16T22:16:46.474629+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:16:46.474629+0000","last_undegraded":"2024-09-16T22:16:46.474629+0000","last_fullsized":"2024-09-16T22:16:46.474629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:58.355316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065480300000000003,"stat_sum":{"num_bytes":8007175,"num_objects":48,"num_object_clones":1,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":38,"num_read":688,"num_read_kb":4413,"num_write":104,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276002+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:46.276002+0000","last_peered":"2024-09-16T22:16:46.276002+0000","last_clean":"2024-09-16T22:16:46.276002+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:46.276002+0000","last_undegraded":"2024-09-16T22:16:46.276002+0000","last_fullsized":"2024-09-16T22:16:46.276002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'212","reported_seq":2169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278254+0000","last_change":"2024-09-16T22:16:17.147841+0000","last_active":"2024-09-16T22:16:46.278254+0000","last_peered":"2024-09-16T22:16:46.278254+0000","last_clean":"2024-09-16T22:16:46.278254+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:16:46.278254+0000","last_undegraded":"2024-09-16T22:16:46.278254+0000","last_fullsized":"2024-09-16T22:16:46.278254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:23.882068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053693199999999995,"stat_sum":{"num_bytes":13761474,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":2187,"num_read_kb":6483,"num_write":220,"num_write_kb":3375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294501+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:46.294501+0000","last_peered":"2024-09-16T22:16:46.294501+0000","last_clean":"2024-09-16T22:16:46.294501+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:46.294501+0000","last_undegraded":"2024-09-16T22:16:46.294501+0000","last_fullsized":"2024-09-16T22:16:46.294501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"47'135","reported_seq":1395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275868+0000","last_change":"2024-09-16T22:16:17.126856+0000","last_active":"2024-09-16T22:16:46.275868+0000","last_peered":"2024-09-16T22:16:46.275868+0000","last_clean":"2024-09-16T22:16:46.275868+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:16:46.275868+0000","last_undegraded":"2024-09-16T22:16:46.275868+0000","last_fullsized":"2024-09-16T22:16:46.275868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:44.976084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066546900000000002,"stat_sum":{"num_bytes":28609961,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":1453,"num_read_kb":14700,"num_write":98,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"47'208","reported_seq":877,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417895+0000","last_change":"2024-09-16T22:16:17.136232+0000","last_active":"2024-09-16T22:16:46.417895+0000","last_peered":"2024-09-16T22:16:46.417895+0000","last_clean":"2024-09-16T22:16:46.417895+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:16:46.417895+0000","last_undegraded":"2024-09-16T22:16:46.417895+0000","last_fullsized":"2024-09-16T22:16:46.417895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:50.781472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078718899999999999,"stat_sum":{"num_bytes":10561849,"num_objects":51,"num_object_clones":4,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":40,"num_read":1092,"num_read_kb":4610,"num_write":195,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275851+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:46.275851+0000","last_peered":"2024-09-16T22:16:46.275851+0000","last_clean":"2024-09-16T22:16:46.275851+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:46.275851+0000","last_undegraded":"2024-09-16T22:16:46.275851+0000","last_fullsized":"2024-09-16T22:16:46.275851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"49'126","reported_seq":454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.437832+0000","last_change":"2024-09-16T22:16:17.136580+0000","last_active":"2024-09-16T22:16:46.437832+0000","last_peered":"2024-09-16T22:16:46.437832+0000","last_clean":"2024-09-16T22:16:46.437832+0000","last_became_active":"2024-09-16T22:16:15.126826+0000","last_became_peered":"2024-09-16T22:16:15.126826+0000","last_unstale":"2024-09-16T22:16:46.437832+0000","last_undegraded":"2024-09-16T22:16:46.437832+0000","last_fullsized":"2024-09-16T22:16:46.437832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:45.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089698499999999999,"stat_sum":{"num_bytes":1989454,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":23,"num_read":675,"num_read_kb":545,"num_write":99,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.337943+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:49.337943+0000","last_peered":"2024-09-16T22:16:49.337943+0000","last_clean":"2024-09-16T22:16:49.337943+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:49.337943+0000","last_undegraded":"2024-09-16T22:16:49.337943+0000","last_fullsized":"2024-09-16T22:16:49.337943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"49'115","reported_seq":779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.607632+0000","last_change":"2024-09-16T22:16:46.346779+0000","last_active":"2024-09-16T22:16:46.607632+0000","last_peered":"2024-09-16T22:16:46.607632+0000","last_clean":"2024-09-16T22:16:46.607632+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:16:46.607632+0000","last_undegraded":"2024-09-16T22:16:46.607632+0000","last_fullsized":"2024-09-16T22:16:46.607632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:56.705809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045343400000000001,"stat_sum":{"num_bytes":8221931,"num_objects":38,"num_object_clones":1,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":27,"num_read":571,"num_read_kb":2851,"num_write":101,"num_write_kb":713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275587+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:46.275587+0000","last_peered":"2024-09-16T22:16:46.275587+0000","last_clean":"2024-09-16T22:16:46.275587+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:46.275587+0000","last_undegraded":"2024-09-16T22:16:46.275587+0000","last_fullsized":"2024-09-16T22:16:46.275587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"47'132","reported_seq":484,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280133+0000","last_change":"2024-09-16T22:16:17.263750+0000","last_active":"2024-09-16T22:16:46.280133+0000","last_peered":"2024-09-16T22:16:46.280133+0000","last_clean":"2024-09-16T22:16:46.280133+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:16:46.280133+0000","last_undegraded":"2024-09-16T22:16:46.280133+0000","last_fullsized":"2024-09-16T22:16:46.280133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:14.235622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060679099999999997,"stat_sum":{"num_bytes":13313472,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":34,"num_read":338,"num_read_kb":14345,"num_write":99,"num_write_kb":10505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275736+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:46.275736+0000","last_peered":"2024-09-16T22:16:46.275736+0000","last_clean":"2024-09-16T22:16:46.275736+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:46.275736+0000","last_undegraded":"2024-09-16T22:16:46.275736+0000","last_fullsized":"2024-09-16T22:16:46.275736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":164,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276204+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:46.276204+0000","last_peered":"2024-09-16T22:16:46.276204+0000","last_clean":"2024-09-16T22:16:46.276204+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:46.276204+0000","last_undegraded":"2024-09-16T22:16:46.276204+0000","last_fullsized":"2024-09-16T22:16:46.276204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"49'172","reported_seq":663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280121+0000","last_change":"2024-09-16T22:16:17.263479+0000","last_active":"2024-09-16T22:16:46.280121+0000","last_peered":"2024-09-16T22:16:46.280121+0000","last_clean":"2024-09-16T22:16:46.280121+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:16:46.280121+0000","last_undegraded":"2024-09-16T22:16:46.280121+0000","last_fullsized":"2024-09-16T22:16:46.280121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:06.292129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450018,"stat_sum":{"num_bytes":4987868,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":35,"num_read":868,"num_read_kb":1843,"num_write":131,"num_write_kb":787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"49'262","reported_seq":814,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560388+0000","last_change":"2024-09-16T22:16:17.128500+0000","last_active":"2024-09-16T22:16:46.560388+0000","last_peered":"2024-09-16T22:16:46.560388+0000","last_clean":"2024-09-16T22:16:46.560388+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:16:46.560388+0000","last_undegraded":"2024-09-16T22:16:46.560388+0000","last_fullsized":"2024-09-16T22:16:46.560388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:42.030243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001733109,"stat_sum":{"num_bytes":7657856,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":1250,"num_read_kb":3842,"num_write":331,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"49'301","reported_seq":680,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275807+0000","last_change":"2024-09-16T22:16:17.127879+0000","last_active":"2024-09-16T22:16:46.275807+0000","last_peered":"2024-09-16T22:16:46.275807+0000","last_clean":"2024-09-16T22:16:46.275807+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:16:46.275807+0000","last_undegraded":"2024-09-16T22:16:46.275807+0000","last_fullsized":"2024-09-16T22:16:46.275807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:53.972422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001383212,"stat_sum":{"num_bytes":9549839,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":959,"num_read_kb":4045,"num_write":347,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"52'206","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.357011+0000","last_change":"2024-09-16T22:16:17.263197+0000","last_active":"2024-09-16T22:16:49.357011+0000","last_peered":"2024-09-16T22:16:49.357011+0000","last_clean":"2024-09-16T22:16:49.357011+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:16:49.357011+0000","last_undegraded":"2024-09-16T22:16:49.357011+0000","last_fullsized":"2024-09-16T22:16:49.357011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:52:46.225583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476284,"stat_sum":{"num_bytes":16326061,"num_objects":58,"num_object_clones":4,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":38,"num_read":1422,"num_read_kb":7919,"num_write":178,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"47'132","reported_seq":606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417902+0000","last_change":"2024-09-16T22:16:17.135988+0000","last_active":"2024-09-16T22:16:46.417902+0000","last_peered":"2024-09-16T22:16:46.417902+0000","last_clean":"2024-09-16T22:16:46.417902+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:16:46.417902+0000","last_undegraded":"2024-09-16T22:16:46.417902+0000","last_fullsized":"2024-09-16T22:16:46.417902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:22.321858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062643099999999999,"stat_sum":{"num_bytes":19277734,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":443,"num_read_kb":4970,"num_write":92,"num_write_kb":2578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"47'204","reported_seq":854,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.480721+0000","last_change":"2024-09-16T22:16:17.128860+0000","last_active":"2024-09-16T22:16:46.480721+0000","last_peered":"2024-09-16T22:16:46.480721+0000","last_clean":"2024-09-16T22:16:46.480721+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:16:46.480721+0000","last_undegraded":"2024-09-16T22:16:46.480721+0000","last_fullsized":"2024-09-16T22:16:46.480721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:43.259469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020238500000000002,"stat_sum":{"num_bytes":1915593,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":29,"num_read":1553,"num_read_kb":1537,"num_write":176,"num_write_kb":4702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'136","reported_seq":672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276233+0000","last_change":"2024-09-16T22:16:17.127452+0000","last_active":"2024-09-16T22:16:46.276233+0000","last_peered":"2024-09-16T22:16:46.276233+0000","last_clean":"2024-09-16T22:16:46.276233+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:16:46.276233+0000","last_undegraded":"2024-09-16T22:16:46.276233+0000","last_fullsized":"2024-09-16T22:16:46.276233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:43.537885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090449699999999998,"stat_sum":{"num_bytes":11838195,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":27,"num_read":504,"num_read_kb":13038,"num_write":110,"num_write_kb":15147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"47'126","reported_seq":460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418041+0000","last_change":"2024-09-16T22:16:17.135885+0000","last_active":"2024-09-16T22:16:46.418041+0000","last_peered":"2024-09-16T22:16:46.418041+0000","last_clean":"2024-09-16T22:16:46.418041+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:16:46.418041+0000","last_undegraded":"2024-09-16T22:16:46.418041+0000","last_fullsized":"2024-09-16T22:16:46.418041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.071537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050898299999999996,"stat_sum":{"num_bytes":3699813,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":35,"num_read":508,"num_read_kb":445,"num_write":89,"num_write_kb":363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"47'161","reported_seq":502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276141+0000","last_change":"2024-09-16T22:16:17.131207+0000","last_active":"2024-09-16T22:16:46.276141+0000","last_peered":"2024-09-16T22:16:46.276141+0000","last_clean":"2024-09-16T22:16:46.276141+0000","last_became_active":"2024-09-16T22:16:15.127331+0000","last_became_peered":"2024-09-16T22:16:15.127331+0000","last_unstale":"2024-09-16T22:16:46.276141+0000","last_undegraded":"2024-09-16T22:16:46.276141+0000","last_fullsized":"2024-09-16T22:16:46.276141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:17.655441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055399799999999999,"stat_sum":{"num_bytes":4041720,"num_objects":39,"num_object_clones":3,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":32,"num_read":605,"num_read_kb":2847,"num_write":169,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"47'422","reported_seq":1669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275589+0000","last_change":"2024-09-16T22:16:17.126205+0000","last_active":"2024-09-16T22:16:46.275589+0000","last_peered":"2024-09-16T22:16:46.275589+0000","last_clean":"2024-09-16T22:16:46.275589+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:16:46.275589+0000","last_undegraded":"2024-09-16T22:16:46.275589+0000","last_fullsized":"2024-09-16T22:16:46.275589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:02.069887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049330399999999996,"stat_sum":{"num_bytes":14962163,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":36,"num_read":3070,"num_read_kb":12486,"num_write":513,"num_write_kb":3434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"48'160","reported_seq":479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278240+0000","last_change":"2024-09-16T22:16:17.147648+0000","last_active":"2024-09-16T22:16:46.278240+0000","last_peered":"2024-09-16T22:16:46.278240+0000","last_clean":"2024-09-16T22:16:46.278240+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:16:46.278240+0000","last_undegraded":"2024-09-16T22:16:46.278240+0000","last_fullsized":"2024-09-16T22:16:46.278240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:55.631170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044714499999999998,"stat_sum":{"num_bytes":11391311,"num_objects":45,"num_object_clones":2,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":38,"num_read":675,"num_read_kb":936,"num_write":128,"num_write_kb":2582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"47'165","reported_seq":665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.464753+0000","last_change":"2024-09-16T22:16:17.127621+0000","last_active":"2024-09-16T22:16:46.464753+0000","last_peered":"2024-09-16T22:16:46.464753+0000","last_clean":"2024-09-16T22:16:46.464753+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:16:46.464753+0000","last_undegraded":"2024-09-16T22:16:46.464753+0000","last_fullsized":"2024-09-16T22:16:46.464753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:32.968348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682444,"stat_sum":{"num_bytes":4535111,"num_objects":41,"num_object_clones":1,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":1305,"num_read_kb":1200,"num_write":132,"num_write_kb":4014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":325759492,"num_objects":1341,"num_object_clones":81,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1341,"num_whiteouts":992,"num_read":47772,"num_read_kb":173341,"num_write":6349,"num_write_kb":130057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":80510976,"data_stored":92499477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6306,"ondisk_log_size":6306,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44096,"kb_used_data":8748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327744,"statfs":{"total":96636764160,"available":96591609856,"internally_reserved":0,"allocated":8957952,"data_stored":12264716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42856,"kb_used_data":7500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328984,"statfs":{"total":96636764160,"available":96592879616,"internally_reserved":0,"allocated":7680000,"data_stored":7234675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49476,"kb_used_data":14124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322364,"statfs":{"total":96636764160,"available":96586100736,"internally_reserved":0,"allocated":14462976,"data_stored":15406271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41696,"kb_used_data":6348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330144,"statfs":{"total":96636764160,"available":96594067456,"internally_reserved":0,"allocated":6500352,"data_stored":7396383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":24280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":24862720,"data_stored":27700733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39040,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332800,"statfs":{"total":96636764160,"available":96596787200,"internally_reserved":0,"allocated":3772416,"data_stored":3296049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43260,"kb_used_data":7912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328580,"statfs":{"total":96636764160,"available":96592465920,"internally_reserved":0,"allocated":8101888,"data_stored":8130735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42408,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329432,"statfs":{"total":96636764160,"available":96593338368,"internally_reserved":0,"allocated":11485184,"data_stored":13921462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7606272,"data_stored":7938493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":2472825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24494080,"data_stored":27612864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6049792,"data_stored":7246016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13754368,"data_stored":15028696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":6411451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8417280,"data_stored":12032109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11005952,"data_stored":13757023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:50.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2395 trunc from 0xe354b33 to 0x4e61ca3 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2396 trunc from 0x4e61ca3 to 0xb16a4 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2397 trunc from 0xb16a4 to 0x6f60387 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2398 punch from 0x22e8fd2 to 0x22eec32, (0x5c60 bytes) 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2399 read 0x5482ade thru 0x5484213 (0x1736 bytes) 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:2400 read 0x210e115 thru 0x2112e53 (0x4d3f bytes) 2024-09-16T22:16:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2402 read 0x51e61bc thru 0x51ee263 (0x80a8 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2403 write 0x290404d thru 0x290730c (0x32c0 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2405 read 0x113c6c7 thru 0x1145ee0 (0x981a bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2406 read 0x3180eaa thru 0x31892aa (0x8401 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2407 write 0x1377bc thru 0x13f2b1 (0x7af6 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2408 write 0x49485c7 thru 0x4948706 (0x140 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2409 write 0xa9abd8 thru 0xa9cc5e (0x2087 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2410 writesame 0x6a806b6 thru 0x6a85165 data_size 0x50(0x4ab0 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2411 write 0x30a20d8 thru 0x30ae468 (0xc391 bytes) 2024-09-16T22:16:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:2412 writesame 0x4b8b9c6 thru 0x4b98599 data_size 0xa(0xcbd4 bytes) 2024-09-16T22:16:50.927 INFO:teuthology.orchestra.run.smithi031.stdout:2413 read 0x2167657 thru 0x216e970 (0x731a bytes) 2024-09-16T22:16:50.927 INFO:teuthology.orchestra.run.smithi031.stdout:2414 write 0x38a1f0c thru 0x38b1086 (0xf17b bytes) 2024-09-16T22:16:50.927 INFO:teuthology.orchestra.run.smithi031.stdout:2416 clone 23 order 24 su 8388608 sc 5 2024-09-16T22:16:51.017 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:51.018+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:51.059 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:51.060 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:51.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:51.088 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:51.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:51.118 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:51.119+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:51.135 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:51.135 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:51.151 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:51.168 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:51.171 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.171 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:51.185 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:51.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:51.218 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:51.218 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:51.220+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:51.276 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:51.283 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.283 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.289 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-16T22:16:50.575739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326349879,"num_objects":1345,"num_object_clones":81,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":992,"num_read":47838,"num_read_kb":173396,"num_write":6452,"num_write_kb":131687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6381,"ondisk_log_size":6381,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":362720,"kb_used_data":83812,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754612000,"statfs":{"total":773094113280,"available":772722688000,"internally_reserved":0,"allocated":85823488,"data_stored":95351024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11041751,"num_objects":419,"num_object_clones":15,"num_object_copies":838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":419,"num_whiteouts":368,"num_read":6154,"num_read_kb":29361,"num_write":1378,"num_write_kb":7592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":144719,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000668"},"pg_stats":[{"pgid":"3.1f","version":"49'224","reported_seq":1751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.566998+0000","last_change":"2024-09-16T22:16:17.136383+0000","last_active":"2024-09-16T22:16:46.566998+0000","last_peered":"2024-09-16T22:16:46.566998+0000","last_clean":"2024-09-16T22:16:46.566998+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:16:46.566998+0000","last_undegraded":"2024-09-16T22:16:46.566998+0000","last_fullsized":"2024-09-16T22:16:46.566998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:37.254425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095346500000000002,"stat_sum":{"num_bytes":11999249,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":34,"num_read":2502,"num_read_kb":5938,"num_write":194,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"47'108","reported_seq":512,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294429+0000","last_change":"2024-09-16T22:16:17.136432+0000","last_active":"2024-09-16T22:16:46.294429+0000","last_peered":"2024-09-16T22:16:46.294429+0000","last_clean":"2024-09-16T22:16:46.294429+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:16:46.294429+0000","last_undegraded":"2024-09-16T22:16:46.294429+0000","last_fullsized":"2024-09-16T22:16:46.294429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:04.096142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078321800000000002,"stat_sum":{"num_bytes":7084633,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":26,"num_read":566,"num_read_kb":12322,"num_write":84,"num_write_kb":30982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"49'146","reported_seq":829,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.568620+0000","last_change":"2024-09-16T22:16:17.128822+0000","last_active":"2024-09-16T22:16:46.568620+0000","last_peered":"2024-09-16T22:16:46.568620+0000","last_clean":"2024-09-16T22:16:46.568620+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:16:46.568620+0000","last_undegraded":"2024-09-16T22:16:46.568620+0000","last_fullsized":"2024-09-16T22:16:46.568620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:04.455146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019949680000000002,"stat_sum":{"num_bytes":6518353,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":19,"num_read":1088,"num_read_kb":1983,"num_write":125,"num_write_kb":982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"48'131","reported_seq":1225,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.467202+0000","last_change":"2024-09-16T22:16:17.131219+0000","last_active":"2024-09-16T22:16:46.467202+0000","last_peered":"2024-09-16T22:16:46.467202+0000","last_clean":"2024-09-16T22:16:46.467202+0000","last_became_active":"2024-09-16T22:16:15.135556+0000","last_became_peered":"2024-09-16T22:16:15.135556+0000","last_unstale":"2024-09-16T22:16:46.467202+0000","last_undegraded":"2024-09-16T22:16:46.467202+0000","last_fullsized":"2024-09-16T22:16:46.467202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:26:53.306823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052703799999999996,"stat_sum":{"num_bytes":4523322,"num_objects":44,"num_object_clones":3,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":33,"num_read":1119,"num_read_kb":4057,"num_write":99,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"49'212","reported_seq":903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.528025+0000","last_change":"2024-09-16T22:16:17.263803+0000","last_active":"2024-09-16T22:16:46.528025+0000","last_peered":"2024-09-16T22:16:46.528025+0000","last_clean":"2024-09-16T22:16:46.528025+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:16:46.528025+0000","last_undegraded":"2024-09-16T22:16:46.528025+0000","last_fullsized":"2024-09-16T22:16:46.528025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:31.454818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053437600000000004,"stat_sum":{"num_bytes":8276329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":25,"num_read":1710,"num_read_kb":1644,"num_write":183,"num_write_kb":1779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"49'164","reported_seq":5619,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.602300+0000","last_change":"2024-09-16T22:16:17.131812+0000","last_active":"2024-09-16T22:16:46.602300+0000","last_peered":"2024-09-16T22:16:46.602300+0000","last_clean":"2024-09-16T22:16:46.602300+0000","last_became_active":"2024-09-16T22:16:15.126160+0000","last_became_peered":"2024-09-16T22:16:15.126160+0000","last_unstale":"2024-09-16T22:16:46.602300+0000","last_undegraded":"2024-09-16T22:16:46.602300+0000","last_fullsized":"2024-09-16T22:16:46.602300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:52.480013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010623589999999999,"stat_sum":{"num_bytes":15445043,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":5525,"num_read_kb":5598,"num_write":131,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"48'99","reported_seq":384,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.425255+0000","last_change":"2024-09-16T22:16:17.263495+0000","last_active":"2024-09-16T22:16:46.425255+0000","last_peered":"2024-09-16T22:16:46.425255+0000","last_clean":"2024-09-16T22:16:46.425255+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:16:46.425255+0000","last_undegraded":"2024-09-16T22:16:46.425255+0000","last_fullsized":"2024-09-16T22:16:46.425255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:22.625830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040510500000000001,"stat_sum":{"num_bytes":9877981,"num_objects":39,"num_object_clones":2,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":27,"num_read":257,"num_read_kb":898,"num_write":69,"num_write_kb":422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"49'210","reported_seq":314,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294526+0000","last_change":"2024-09-16T22:16:17.136712+0000","last_active":"2024-09-16T22:16:46.294526+0000","last_peered":"2024-09-16T22:16:46.294526+0000","last_clean":"2024-09-16T22:16:46.294526+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:16:46.294526+0000","last_undegraded":"2024-09-16T22:16:46.294526+0000","last_fullsized":"2024-09-16T22:16:46.294526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:52.131127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096381500000000003,"stat_sum":{"num_bytes":4589609,"num_objects":37,"num_object_clones":1,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":31,"num_read":592,"num_read_kb":616,"num_write":252,"num_write_kb":563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"49'388","reported_seq":1079,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294610+0000","last_change":"2024-09-16T22:16:17.135852+0000","last_active":"2024-09-16T22:16:46.294610+0000","last_peered":"2024-09-16T22:16:46.294610+0000","last_clean":"2024-09-16T22:16:46.294610+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:16:46.294610+0000","last_undegraded":"2024-09-16T22:16:46.294610+0000","last_fullsized":"2024-09-16T22:16:46.294610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:30.009725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042473899999999999,"stat_sum":{"num_bytes":15147821,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":29,"num_read":2047,"num_read_kb":8519,"num_write":564,"num_write_kb":17525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"47'137","reported_seq":1407,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276133+0000","last_change":"2024-09-16T22:16:17.127257+0000","last_active":"2024-09-16T22:16:46.276133+0000","last_peered":"2024-09-16T22:16:46.276133+0000","last_clean":"2024-09-16T22:16:46.276133+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:16:46.276133+0000","last_undegraded":"2024-09-16T22:16:46.276133+0000","last_fullsized":"2024-09-16T22:16:46.276133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:13.884897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093962400000000001,"stat_sum":{"num_bytes":13638927,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":1231,"num_read_kb":6032,"num_write":111,"num_write_kb":2031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"49'401","reported_seq":796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418090+0000","last_change":"2024-09-16T22:16:17.135778+0000","last_active":"2024-09-16T22:16:46.418090+0000","last_peered":"2024-09-16T22:16:46.418090+0000","last_clean":"2024-09-16T22:16:46.418090+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:16:46.418090+0000","last_undegraded":"2024-09-16T22:16:46.418090+0000","last_fullsized":"2024-09-16T22:16:46.418090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:36.889739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041336299999999998,"stat_sum":{"num_bytes":12518884,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":1765,"num_read_kb":7416,"num_write":558,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"49'144","reported_seq":448,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560653+0000","last_change":"2024-09-16T22:16:17.128475+0000","last_active":"2024-09-16T22:16:46.560653+0000","last_peered":"2024-09-16T22:16:46.560653+0000","last_clean":"2024-09-16T22:16:46.560653+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:16:46.560653+0000","last_undegraded":"2024-09-16T22:16:46.560653+0000","last_fullsized":"2024-09-16T22:16:46.560653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:42.401536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001932436,"stat_sum":{"num_bytes":9306351,"num_objects":36,"num_object_clones":1,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":28,"num_read":391,"num_read_kb":1711,"num_write":135,"num_write_kb":531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275996+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:46.275996+0000","last_peered":"2024-09-16T22:16:46.275996+0000","last_clean":"2024-09-16T22:16:46.275996+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:46.275996+0000","last_undegraded":"2024-09-16T22:16:46.275996+0000","last_fullsized":"2024-09-16T22:16:46.275996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"49'422","reported_seq":6951,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.482335+0000","last_change":"2024-09-16T22:16:17.263952+0000","last_active":"2024-09-16T22:16:46.482335+0000","last_peered":"2024-09-16T22:16:46.482335+0000","last_clean":"2024-09-16T22:16:46.482335+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:16:46.482335+0000","last_undegraded":"2024-09-16T22:16:46.482335+0000","last_fullsized":"2024-09-16T22:16:46.482335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:53.349620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063812899999999997,"stat_sum":{"num_bytes":12184410,"num_objects":44,"num_object_clones":2,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":8813,"num_read_kb":13552,"num_write":532,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280213+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:46.280213+0000","last_peered":"2024-09-16T22:16:46.280213+0000","last_clean":"2024-09-16T22:16:46.280213+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:46.280213+0000","last_undegraded":"2024-09-16T22:16:46.280213+0000","last_fullsized":"2024-09-16T22:16:46.280213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"49'145","reported_seq":633,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.474629+0000","last_change":"2024-09-16T22:16:17.264006+0000","last_active":"2024-09-16T22:16:46.474629+0000","last_peered":"2024-09-16T22:16:46.474629+0000","last_clean":"2024-09-16T22:16:46.474629+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:16:46.474629+0000","last_undegraded":"2024-09-16T22:16:46.474629+0000","last_fullsized":"2024-09-16T22:16:46.474629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:58.355316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065480300000000003,"stat_sum":{"num_bytes":8007175,"num_objects":48,"num_object_clones":1,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":38,"num_read":688,"num_read_kb":4413,"num_write":104,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276002+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:46.276002+0000","last_peered":"2024-09-16T22:16:46.276002+0000","last_clean":"2024-09-16T22:16:46.276002+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:46.276002+0000","last_undegraded":"2024-09-16T22:16:46.276002+0000","last_fullsized":"2024-09-16T22:16:46.276002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'212","reported_seq":2169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278254+0000","last_change":"2024-09-16T22:16:17.147841+0000","last_active":"2024-09-16T22:16:46.278254+0000","last_peered":"2024-09-16T22:16:46.278254+0000","last_clean":"2024-09-16T22:16:46.278254+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:16:46.278254+0000","last_undegraded":"2024-09-16T22:16:46.278254+0000","last_fullsized":"2024-09-16T22:16:46.278254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:23.882068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053693199999999995,"stat_sum":{"num_bytes":13761474,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":2187,"num_read_kb":6483,"num_write":220,"num_write_kb":3375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294501+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:46.294501+0000","last_peered":"2024-09-16T22:16:46.294501+0000","last_clean":"2024-09-16T22:16:46.294501+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:46.294501+0000","last_undegraded":"2024-09-16T22:16:46.294501+0000","last_fullsized":"2024-09-16T22:16:46.294501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"47'135","reported_seq":1395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275868+0000","last_change":"2024-09-16T22:16:17.126856+0000","last_active":"2024-09-16T22:16:46.275868+0000","last_peered":"2024-09-16T22:16:46.275868+0000","last_clean":"2024-09-16T22:16:46.275868+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:16:46.275868+0000","last_undegraded":"2024-09-16T22:16:46.275868+0000","last_fullsized":"2024-09-16T22:16:46.275868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:44.976084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066546900000000002,"stat_sum":{"num_bytes":28609961,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":1453,"num_read_kb":14700,"num_write":98,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"47'208","reported_seq":877,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417895+0000","last_change":"2024-09-16T22:16:17.136232+0000","last_active":"2024-09-16T22:16:46.417895+0000","last_peered":"2024-09-16T22:16:46.417895+0000","last_clean":"2024-09-16T22:16:46.417895+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:16:46.417895+0000","last_undegraded":"2024-09-16T22:16:46.417895+0000","last_fullsized":"2024-09-16T22:16:46.417895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:50.781472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078718899999999999,"stat_sum":{"num_bytes":10561849,"num_objects":51,"num_object_clones":4,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":40,"num_read":1092,"num_read_kb":4610,"num_write":195,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275851+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:46.275851+0000","last_peered":"2024-09-16T22:16:46.275851+0000","last_clean":"2024-09-16T22:16:46.275851+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:46.275851+0000","last_undegraded":"2024-09-16T22:16:46.275851+0000","last_fullsized":"2024-09-16T22:16:46.275851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"49'126","reported_seq":454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.437832+0000","last_change":"2024-09-16T22:16:17.136580+0000","last_active":"2024-09-16T22:16:46.437832+0000","last_peered":"2024-09-16T22:16:46.437832+0000","last_clean":"2024-09-16T22:16:46.437832+0000","last_became_active":"2024-09-16T22:16:15.126826+0000","last_became_peered":"2024-09-16T22:16:15.126826+0000","last_unstale":"2024-09-16T22:16:46.437832+0000","last_undegraded":"2024-09-16T22:16:46.437832+0000","last_fullsized":"2024-09-16T22:16:46.437832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:45.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089698499999999999,"stat_sum":{"num_bytes":1989454,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":23,"num_read":675,"num_read_kb":545,"num_write":99,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.337943+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:49.337943+0000","last_peered":"2024-09-16T22:16:49.337943+0000","last_clean":"2024-09-16T22:16:49.337943+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:49.337943+0000","last_undegraded":"2024-09-16T22:16:49.337943+0000","last_fullsized":"2024-09-16T22:16:49.337943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"49'115","reported_seq":779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.607632+0000","last_change":"2024-09-16T22:16:46.346779+0000","last_active":"2024-09-16T22:16:46.607632+0000","last_peered":"2024-09-16T22:16:46.607632+0000","last_clean":"2024-09-16T22:16:46.607632+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:16:46.607632+0000","last_undegraded":"2024-09-16T22:16:46.607632+0000","last_fullsized":"2024-09-16T22:16:46.607632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:56.705809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045343400000000001,"stat_sum":{"num_bytes":8221931,"num_objects":38,"num_object_clones":1,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":27,"num_read":571,"num_read_kb":2851,"num_write":101,"num_write_kb":713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275587+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:46.275587+0000","last_peered":"2024-09-16T22:16:46.275587+0000","last_clean":"2024-09-16T22:16:46.275587+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:46.275587+0000","last_undegraded":"2024-09-16T22:16:46.275587+0000","last_fullsized":"2024-09-16T22:16:46.275587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"47'132","reported_seq":484,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280133+0000","last_change":"2024-09-16T22:16:17.263750+0000","last_active":"2024-09-16T22:16:46.280133+0000","last_peered":"2024-09-16T22:16:46.280133+0000","last_clean":"2024-09-16T22:16:46.280133+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:16:46.280133+0000","last_undegraded":"2024-09-16T22:16:46.280133+0000","last_fullsized":"2024-09-16T22:16:46.280133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:14.235622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060679099999999997,"stat_sum":{"num_bytes":13313472,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":34,"num_read":338,"num_read_kb":14345,"num_write":99,"num_write_kb":10505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275736+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:46.275736+0000","last_peered":"2024-09-16T22:16:46.275736+0000","last_clean":"2024-09-16T22:16:46.275736+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:46.275736+0000","last_undegraded":"2024-09-16T22:16:46.275736+0000","last_fullsized":"2024-09-16T22:16:46.275736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":164,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276204+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:46.276204+0000","last_peered":"2024-09-16T22:16:46.276204+0000","last_clean":"2024-09-16T22:16:46.276204+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:46.276204+0000","last_undegraded":"2024-09-16T22:16:46.276204+0000","last_fullsized":"2024-09-16T22:16:46.276204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"49'172","reported_seq":663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280121+0000","last_change":"2024-09-16T22:16:17.263479+0000","last_active":"2024-09-16T22:16:46.280121+0000","last_peered":"2024-09-16T22:16:46.280121+0000","last_clean":"2024-09-16T22:16:46.280121+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:16:46.280121+0000","last_undegraded":"2024-09-16T22:16:46.280121+0000","last_fullsized":"2024-09-16T22:16:46.280121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:06.292129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450018,"stat_sum":{"num_bytes":4987868,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":35,"num_read":868,"num_read_kb":1843,"num_write":131,"num_write_kb":787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"49'262","reported_seq":814,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560388+0000","last_change":"2024-09-16T22:16:17.128500+0000","last_active":"2024-09-16T22:16:46.560388+0000","last_peered":"2024-09-16T22:16:46.560388+0000","last_clean":"2024-09-16T22:16:46.560388+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:16:46.560388+0000","last_undegraded":"2024-09-16T22:16:46.560388+0000","last_fullsized":"2024-09-16T22:16:46.560388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:42.030243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001733109,"stat_sum":{"num_bytes":7657856,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":1250,"num_read_kb":3842,"num_write":331,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"49'301","reported_seq":680,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275807+0000","last_change":"2024-09-16T22:16:17.127879+0000","last_active":"2024-09-16T22:16:46.275807+0000","last_peered":"2024-09-16T22:16:46.275807+0000","last_clean":"2024-09-16T22:16:46.275807+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:16:46.275807+0000","last_undegraded":"2024-09-16T22:16:46.275807+0000","last_fullsized":"2024-09-16T22:16:46.275807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:53.972422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001383212,"stat_sum":{"num_bytes":9549839,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":959,"num_read_kb":4045,"num_write":347,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"52'206","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.357011+0000","last_change":"2024-09-16T22:16:17.263197+0000","last_active":"2024-09-16T22:16:49.357011+0000","last_peered":"2024-09-16T22:16:49.357011+0000","last_clean":"2024-09-16T22:16:49.357011+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:16:49.357011+0000","last_undegraded":"2024-09-16T22:16:49.357011+0000","last_fullsized":"2024-09-16T22:16:49.357011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:52:46.225583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476284,"stat_sum":{"num_bytes":16326061,"num_objects":58,"num_object_clones":4,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":38,"num_read":1422,"num_read_kb":7919,"num_write":178,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"47'132","reported_seq":606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417902+0000","last_change":"2024-09-16T22:16:17.135988+0000","last_active":"2024-09-16T22:16:46.417902+0000","last_peered":"2024-09-16T22:16:46.417902+0000","last_clean":"2024-09-16T22:16:46.417902+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:16:46.417902+0000","last_undegraded":"2024-09-16T22:16:46.417902+0000","last_fullsized":"2024-09-16T22:16:46.417902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:22.321858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062643099999999999,"stat_sum":{"num_bytes":19277734,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":443,"num_read_kb":4970,"num_write":92,"num_write_kb":2578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"47'204","reported_seq":854,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.480721+0000","last_change":"2024-09-16T22:16:17.128860+0000","last_active":"2024-09-16T22:16:46.480721+0000","last_peered":"2024-09-16T22:16:46.480721+0000","last_clean":"2024-09-16T22:16:46.480721+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:16:46.480721+0000","last_undegraded":"2024-09-16T22:16:46.480721+0000","last_fullsized":"2024-09-16T22:16:46.480721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:43.259469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020238500000000002,"stat_sum":{"num_bytes":1915593,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":29,"num_read":1553,"num_read_kb":1537,"num_write":176,"num_write_kb":4702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'136","reported_seq":672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276233+0000","last_change":"2024-09-16T22:16:17.127452+0000","last_active":"2024-09-16T22:16:46.276233+0000","last_peered":"2024-09-16T22:16:46.276233+0000","last_clean":"2024-09-16T22:16:46.276233+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:16:46.276233+0000","last_undegraded":"2024-09-16T22:16:46.276233+0000","last_fullsized":"2024-09-16T22:16:46.276233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:43.537885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090449699999999998,"stat_sum":{"num_bytes":11838195,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":27,"num_read":504,"num_read_kb":13038,"num_write":110,"num_write_kb":15147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"47'126","reported_seq":460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418041+0000","last_change":"2024-09-16T22:16:17.135885+0000","last_active":"2024-09-16T22:16:46.418041+0000","last_peered":"2024-09-16T22:16:46.418041+0000","last_clean":"2024-09-16T22:16:46.418041+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:16:46.418041+0000","last_undegraded":"2024-09-16T22:16:46.418041+0000","last_fullsized":"2024-09-16T22:16:46.418041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.071537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050898299999999996,"stat_sum":{"num_bytes":3699813,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":35,"num_read":508,"num_read_kb":445,"num_write":89,"num_write_kb":363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"47'161","reported_seq":502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276141+0000","last_change":"2024-09-16T22:16:17.131207+0000","last_active":"2024-09-16T22:16:46.276141+0000","last_peered":"2024-09-16T22:16:46.276141+0000","last_clean":"2024-09-16T22:16:46.276141+0000","last_became_active":"2024-09-16T22:16:15.127331+0000","last_became_peered":"2024-09-16T22:16:15.127331+0000","last_unstale":"2024-09-16T22:16:46.276141+0000","last_undegraded":"2024-09-16T22:16:46.276141+0000","last_fullsized":"2024-09-16T22:16:46.276141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:17.655441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055399799999999999,"stat_sum":{"num_bytes":4041720,"num_objects":39,"num_object_clones":3,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":32,"num_read":605,"num_read_kb":2847,"num_write":169,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"47'422","reported_seq":1669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275589+0000","last_change":"2024-09-16T22:16:17.126205+0000","last_active":"2024-09-16T22:16:46.275589+0000","last_peered":"2024-09-16T22:16:46.275589+0000","last_clean":"2024-09-16T22:16:46.275589+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:16:46.275589+0000","last_undegraded":"2024-09-16T22:16:46.275589+0000","last_fullsized":"2024-09-16T22:16:46.275589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:02.069887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049330399999999996,"stat_sum":{"num_bytes":14962163,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":36,"num_read":3070,"num_read_kb":12486,"num_write":513,"num_write_kb":3434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"48'160","reported_seq":479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278240+0000","last_change":"2024-09-16T22:16:17.147648+0000","last_active":"2024-09-16T22:16:46.278240+0000","last_peered":"2024-09-16T22:16:46.278240+0000","last_clean":"2024-09-16T22:16:46.278240+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:16:46.278240+0000","last_undegraded":"2024-09-16T22:16:46.278240+0000","last_fullsized":"2024-09-16T22:16:46.278240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:55.631170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044714499999999998,"stat_sum":{"num_bytes":11391311,"num_objects":45,"num_object_clones":2,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":38,"num_read":675,"num_read_kb":936,"num_write":128,"num_write_kb":2582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"47'165","reported_seq":665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.464753+0000","last_change":"2024-09-16T22:16:17.127621+0000","last_active":"2024-09-16T22:16:46.464753+0000","last_peered":"2024-09-16T22:16:46.464753+0000","last_clean":"2024-09-16T22:16:46.464753+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:16:46.464753+0000","last_undegraded":"2024-09-16T22:16:46.464753+0000","last_fullsized":"2024-09-16T22:16:46.464753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:32.968348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682444,"stat_sum":{"num_bytes":4535111,"num_objects":41,"num_object_clones":1,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":1305,"num_read_kb":1200,"num_write":132,"num_write_kb":4014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":325759492,"num_objects":1341,"num_object_clones":81,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1341,"num_whiteouts":992,"num_read":47772,"num_read_kb":173341,"num_write":6349,"num_write_kb":130057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":80510976,"data_stored":92499477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6306,"ondisk_log_size":6306,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44096,"kb_used_data":8748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327744,"statfs":{"total":96636764160,"available":96591609856,"internally_reserved":0,"allocated":8957952,"data_stored":12264716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42856,"kb_used_data":7500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328984,"statfs":{"total":96636764160,"available":96592879616,"internally_reserved":0,"allocated":7680000,"data_stored":7234675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49476,"kb_used_data":14124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322364,"statfs":{"total":96636764160,"available":96586100736,"internally_reserved":0,"allocated":14462976,"data_stored":15406271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41696,"kb_used_data":6348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330144,"statfs":{"total":96636764160,"available":96594067456,"internally_reserved":0,"allocated":6500352,"data_stored":7396383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":24280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":24862720,"data_stored":27700733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39040,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332800,"statfs":{"total":96636764160,"available":96596787200,"internally_reserved":0,"allocated":3772416,"data_stored":3296049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43260,"kb_used_data":7912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328580,"statfs":{"total":96636764160,"available":96592465920,"internally_reserved":0,"allocated":8101888,"data_stored":8130735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42408,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329432,"statfs":{"total":96636764160,"available":96593338368,"internally_reserved":0,"allocated":11485184,"data_stored":13921462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7606272,"data_stored":7938493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":2472825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24494080,"data_stored":27612864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6049792,"data_stored":7246016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13754368,"data_stored":15028696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":6411451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8417280,"data_stored":12032109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11005952,"data_stored":13757023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:51.291 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T22:16:51.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:51.291 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T22:16:51.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T22:16:51.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:51.336 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:51.338+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone22 from 0x6f60387 (overlap 0xb16a4) to 0x58b52 2024-09-16T22:16:51.403 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.404 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:51.420 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:51.421+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T22:16:51.521 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:51.523+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.522 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:51.541 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.541 INFO:teuthology.orchestra.run.smithi031.stdout:2419 punch from 0x64d3fae to 0x64d62a0, (0x22f2 bytes) 2024-09-16T22:16:51.546 INFO:teuthology.orchestra.run.smithi031.stdout:2420 read 0x48da846 thru 0x48ea16b (0xf926 bytes) 2024-09-16T22:16:51.546 INFO:teuthology.orchestra.run.smithi031.stdout:2421 read 0x6bbd93a thru 0x6bbedf5 (0x14bc bytes) 2024-09-16T22:16:51.546 INFO:teuthology.orchestra.run.smithi031.stdout:2423 writesame 0x66ea9dc thru 0x66f8dc0 data_size 0x1(0xe3e5 bytes) 2024-09-16T22:16:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:2428 read 0x677903e thru 0x677ed59 (0x5d1c bytes) 2024-09-16T22:16:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:2429 read 0x14a0f73 thru 0x14a97ea (0x8878 bytes) 2024-09-16T22:16:51.583 INFO:teuthology.orchestra.run.smithi031.stdout:2431 punch from 0x453d8c9 to 0x4541ef8, (0x462f bytes) 2024-09-16T22:16:51.584 INFO:teuthology.orchestra.run.smithi031.stdout:2432 read 0x30a8086 thru 0x30b0442 (0x83bd bytes) 2024-09-16T22:16:51.584 INFO:teuthology.orchestra.run.smithi031.stdout:2433 read 0x5fd451 thru 0x608822 (0xb3d2 bytes) 2024-09-16T22:16:51.584 INFO:teuthology.orchestra.run.smithi031.stdout:2435 writesame 0x5db878b thru 0x5dc6d82 data_size 0x2c(0xe5f8 bytes) 2024-09-16T22:16:51.588 INFO:teuthology.orchestra.run.smithi031.stdout:2436 read 0x58dcfb2 thru 0x58eaa3b (0xda8a bytes) 2024-09-16T22:16:51.603 INFO:teuthology.orchestra.run.smithi031.stdout:2437 write 0xeca5da0 thru 0xeca7f8f (0x21f0 bytes) 2024-09-16T22:16:51.606 INFO:teuthology.orchestra.run.smithi031.stdout:2438 write 0xc25a1b7 thru 0xc26178d (0x75d7 bytes) 2024-09-16T22:16:51.622 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:51.624+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:51.623 INFO:teuthology.orchestra.run.smithi031.stdout:2439 trunc from 0xeca7f90 to 0xa386be 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 54, 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-16T22:15:44.319460+0000", 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-16T22:16:51.308693+0000", 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-16T22:15:56.374983+0000", 2024-09-16T22:16:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-16T22:15:46.438697+0000", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-16T22:16:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:15:56.583719+0000", 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-16T22:16:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:02.234097+0000", 2024-09-16T22:16:51.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:51.646 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "22", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 22, 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-16T22:16:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:13.789572+0000", 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:16:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-16T22:16:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "54", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:16:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 25, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 54, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:16:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:16:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:16:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "44de8de0-e16a-4325-94b9-1cc61f4829db", 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2585124124", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2585124124", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2585124124", 2024-09-16T22:16:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2585124124", 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0e05fbd1-9707-40b8-91a3-198c6f5b7865", 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T22:16:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:16:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3271285223", 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3271285223", 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3271285223", 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3271285223", 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "b080bc29-0a92-4200-ae3f-0a17a1722145", 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2874892035", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2874892035", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2874892035", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2874892035", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0bc64c75-486c-40ac-abaf-381e151623bd", 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3768411773", 2024-09-16T22:16:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3768411773", 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3768411773", 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3768411773", 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fae627d1-6bc2-4e5b-85e1-334e5e05f830", 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6824", 2024-09-16T22:16:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6825", 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6826", 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6827", 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6830", 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6831", 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6828", 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6829", 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6825/3421901083", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6827/3421901083", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3421901083", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6829/3421901083", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "989a79cd-ef84-4bf3-a64f-9d101d0b29d7", 2024-09-16T22:16:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6816", 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6817", 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6818", 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6819", 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6822", 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6823", 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6820", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6821", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6817/335833224", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6819/335833224", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6823/335833224", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6821/335833224", 2024-09-16T22:16:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "92f145b7-e868-43fd-9abe-68c43a407ae2", 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 48, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6808", 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6809", 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6810", 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6811", 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6814", 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6815", 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6812", 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6813", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6809/1592419018", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6811/1592419018", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1592419018", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1592419018", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "373f8002-53a4-4ca7-bbbf-70e8d56b3571", 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:16:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6800", 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6801", 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6802", 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6803", 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6806", 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6807", 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6804", 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6805", 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:16:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6801/3448871901", 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6803/3448871901", 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3448871901", 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3448871901", 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.033837+0000", 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.799099+0000", 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.769970+0000", 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.965029+0000", 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.706036+0000", 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.158188+0000", 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.722931+0000", 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.485009+0000", 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-16T22:16:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-16T22:16:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:51.712 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:51.713 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.713 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.714 INFO:tasks.thrashosds.thrasher:j is {'epoch': 54, 'fsid': '57fe7da5-e356-48a4-ba33-5558b1026a32', 'created': '2024-09-16T22:15:44.319460+0000', 'modified': '2024-09-16T22:16:51.308693+0000', 'last_up_change': '2024-09-16T22:15:56.374983+0000', 'last_in_change': '2024-09-16T22:15:46.438697+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T22:15:56.583719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T22:16:02.234097+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T22:16:13.789572+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 54, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '44de8de0-e16a-4325-94b9-1cc61f4829db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2585124124}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2585124124}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2585124124}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2585124124}]}, 'public_addr': '172.21.15.31:6817/2585124124', 'cluster_addr': '172.21.15.31:6819/2585124124', 'heartbeat_back_addr': '172.21.15.31:6823/2585124124', 'heartbeat_front_addr': '172.21.15.31:6821/2585124124', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3271285223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3271285223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3271285223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3271285223}]}, 'public_addr': '172.21.15.31:6801/3271285223', 'cluster_addr': '172.21.15.31:6803/3271285223', 'heartbeat_back_addr': '172.21.15.31:6807/3271285223', 'heartbeat_front_addr': '172.21.15.31:6805/3271285223', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b080bc29-0a92-4200-ae3f-0a17a1722145', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2874892035}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2874892035}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2874892035}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2874892035}]}, 'public_addr': '172.21.15.31:6809/2874892035', 'cluster_addr': '172.21.15.31:6811/2874892035', 'heartbeat_back_addr': '172.21.15.31:6815/2874892035', 'heartbeat_front_addr': '172.21.15.31:6813/2874892035', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0bc64c75-486c-40ac-abaf-381e151623bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3768411773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3768411773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3768411773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3768411773}]}, 'public_addr': '172.21.15.31:6825/3768411773', 'cluster_addr': '172.21.15.31:6827/3768411773', 'heartbeat_back_addr': '172.21.15.31:6831/3768411773', 'heartbeat_front_addr': '172.21.15.31:6829/3768411773', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 3421901083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3421901083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3421901083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3421901083}]}, 'public_addr': '172.21.15.92:6825/3421901083', 'cluster_addr': '172.21.15.92:6827/3421901083', 'heartbeat_back_addr': '172.21.15.92:6831/3421901083', 'heartbeat_front_addr': '172.21.15.92:6829/3421901083', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 335833224}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 335833224}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 335833224}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 335833224}]}, 'public_addr': '172.21.15.92:6817/335833224', 'cluster_addr': '172.21.15.92:6819/335833224', 'heartbeat_back_addr': '172.21.15.92:6823/335833224', 'heartbeat_front_addr': '172.21.15.92:6821/335833224', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '92f145b7-e868-43fd-9abe-68c43a407ae2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1592419018}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1592419018}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1592419018}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1592419018}]}, 'public_addr': '172.21.15.92:6809/1592419018', 'cluster_addr': '172.21.15.92:6811/1592419018', 'heartbeat_back_addr': '172.21.15.92:6815/1592419018', 'heartbeat_front_addr': '172.21.15.92:6813/1592419018', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '373f8002-53a4-4ca7-bbbf-70e8d56b3571', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3448871901}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3448871901}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3448871901}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3448871901}]}, 'public_addr': '172.21.15.92:6801/3448871901', 'cluster_addr': '172.21.15.92:6803/3448871901', 'heartbeat_back_addr': '172.21.15.92:6807/3448871901', 'heartbeat_front_addr': '172.21.15.92:6805/3448871901', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.033837+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.799099+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.769970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.965029+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.706036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.158188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.722931+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.485009+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T22:16:51.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:16:51.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:2440 write 0x43c05f1 thru 0x43c0857 (0x267 bytes) 2024-09-16T22:16:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:2442 write 0xece71af thru 0xecf5600 (0xe452 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2443 read 0x88854e7 thru 0x888eb0c (0x9626 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2444 punch from 0x8c8ab43 to 0x8c8b459, (0x916 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2445 read 0x4a4b7b3 thru 0x4a4ff78 (0x47c6 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2447 trunc from 0xecf5601 to 0x75b5e67 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2450 read 0x1ae135f thru 0x1ae51f6 (0x3e98 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2451 write 0x802f9f5 thru 0x8032eca (0x34d6 bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2453 writesame 0xa3bbcb7 thru 0xa3c6cb3 data_size 0x1(0xaffd bytes) 2024-09-16T22:16:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:2454 write 0x3376f9e thru 0x3381897 (0xa8fa bytes) 2024-09-16T22:16:51.722 INFO:teuthology.orchestra.run.smithi031.stdout:2456 write 0xb40f9d6 thru 0xb418f9d (0x95c8 bytes) 2024-09-16T22:16:51.725 INFO:teuthology.orchestra.run.smithi031.stdout:2457 punch from 0x9bcce80 to 0x9bd8923, (0xbaa3 bytes) 2024-09-16T22:16:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:2460 write 0xda3b1a8 thru 0xda428f5 (0x774e bytes) 2024-09-16T22:16:51.735 INFO:teuthology.orchestra.run.smithi031.stdout:2461 write 0x95bf1ba thru 0x95c6caa (0x7af1 bytes) 2024-09-16T22:16:51.739 INFO:teuthology.orchestra.run.smithi031.stdout:2462 read 0x3705aea thru 0x37111e4 (0xb6fb bytes) 2024-09-16T22:16:51.739 INFO:teuthology.orchestra.run.smithi031.stdout:2463 read 0x9b06f46 thru 0x9b1292c (0xb9e7 bytes) 2024-09-16T22:16:51.751 INFO:teuthology.orchestra.run.smithi031.stdout:2467 trunc from 0xda428f6 to 0x30a49cd 2024-09-16T22:16:51.754 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:51.756+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:51.770 INFO:teuthology.orchestra.run.smithi031.stdout:2469 writesame 0x5606b12 thru 0x560a927 data_size 0x9(0x3e16 bytes) 2024-09-16T22:16:51.772 INFO:teuthology.orchestra.run.smithi031.stdout:2470 trunc from 0x560a928 to 0x761c585 2024-09-16T22:16:51.791 INFO:teuthology.orchestra.run.smithi031.stdout:2471 writesame 0xe331910 thru 0xe33b7f7 data_size 0x78(0x9ee8 bytes) 2024-09-16T22:16:51.791 INFO:teuthology.orchestra.run.smithi031.stdout:2473 read 0x4bc695 thru 0x4c2192 (0x5afe bytes) 2024-09-16T22:16:51.791 INFO:teuthology.orchestra.run.smithi031.stdout:2475 write 0xab1618e thru 0xab1ab40 (0x49b3 bytes) 2024-09-16T22:16:51.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.794 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.805 INFO:teuthology.orchestra.run.smithi031.stdout:2478 trunc from 0xe33b7f8 to 0x49ab5f5 2024-09-16T22:16:51.823 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:51.825+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:51.826 INFO:teuthology.orchestra.run.smithi031.stdout:2479 write 0xa6e45e3 thru 0xa6e5921 (0x133f bytes) 2024-09-16T22:16:51.833 INFO:teuthology.orchestra.run.smithi031.stdout:2481 trunc from 0xa6e5922 to 0x66bc34b 2024-09-16T22:16:51.844 INFO:teuthology.orchestra.run.smithi031.stdout:2482 punch from 0x2537b53 to 0x2537ff5, (0x4a2 bytes) 2024-09-16T22:16:51.844 INFO:teuthology.orchestra.run.smithi031.stdout:2483 punch from 0x260e3c8 to 0x261498d, (0x65c5 bytes) 2024-09-16T22:16:51.844 INFO:teuthology.orchestra.run.smithi031.stdout:2484 writesame 0x430975e thru 0x4319203 data_size 0x1(0xfaa6 bytes) 2024-09-16T22:16:51.860 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:51.860 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:51.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:51.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.879 INFO:teuthology.orchestra.run.smithi031.stdout:2490 read 0x54ef833 thru 0x54f53b7 (0x5b85 bytes) 2024-09-16T22:16:51.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.888 INFO:teuthology.orchestra.run.smithi031.stdout:2493 writesame 0x92368f0 thru 0x92464bc data_size 0x1(0xfbcd bytes) 2024-09-16T22:16:51.888 INFO:teuthology.orchestra.run.smithi031.stdout:2494 read 0x73a0b3f thru 0x73a7308 (0x67ca bytes) 2024-09-16T22:16:51.889 INFO:teuthology.orchestra.run.smithi031.stdout:2495 punch from 0x612a483 to 0x612ec11, (0x478e bytes) 2024-09-16T22:16:51.897 INFO:teuthology.orchestra.run.smithi031.stdout:2496 writesame 0xbdfda30 thru 0xbe0b15a data_size 0x7(0xd72b bytes) 2024-09-16T22:16:51.903 INFO:teuthology.orchestra.run.smithi031.stdout:2498 writesame 0xc7af502 thru 0xc7b88d6 data_size 0x2d(0x93d5 bytes) 2024-09-16T22:16:51.903 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.909 INFO:teuthology.orchestra.run.smithi031.stdout:2502 write 0xd81415e thru 0xd81d041 (0x8ee4 bytes) 2024-09-16T22:16:51.918 INFO:teuthology.orchestra.run.smithi031.stdout:2503 write 0xe05bb5c thru 0xe061d57 (0x61fc bytes) 2024-09-16T22:16:51.921 INFO:teuthology.orchestra.run.smithi031.stdout:2506 write 0x53c163e thru 0x53c82e8 (0x6cab bytes) 2024-09-16T22:16:51.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.924 INFO:teuthology.orchestra.run.smithi031.stdout:2509 read 0xd7f01f2 thru 0xd7f529a (0x50a9 bytes) 2024-09-16T22:16:51.924 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:51.925+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:51.934 INFO:teuthology.orchestra.run.smithi031.stdout:2511 trunc from 0xe061d58 to 0x5050dd9 2024-09-16T22:16:51.945 INFO:teuthology.orchestra.run.smithi031.stdout:2513 read 0x34f2aa8 thru 0x34f81ae (0x5707 bytes) 2024-09-16T22:16:51.945 INFO:teuthology.orchestra.run.smithi031.stdout:2514 punch from 0x20e0842 to 0x20e64ca, (0x5c88 bytes) 2024-09-16T22:16:51.945 INFO:teuthology.orchestra.run.smithi031.stdout:2516 writesame 0x377ddaa thru 0x378714e data_size 0x2b(0x93a5 bytes) 2024-09-16T22:16:51.962 INFO:teuthology.orchestra.run.smithi031.stdout:2517 write 0xe170d57 thru 0xe176b40 (0x5dea bytes) 2024-09-16T22:16:51.965 INFO:teuthology.orchestra.run.smithi031.stdout:2518 read 0x52f19bb thru 0x5300e88 (0xf4ce bytes) 2024-09-16T22:16:51.965 INFO:teuthology.orchestra.run.smithi031.stdout:2519 write 0x829671c thru 0x82a2079 (0xb95e bytes) 2024-09-16T22:16:51.968 INFO:teuthology.orchestra.run.smithi031.stdout:2523 write 0xd56bbb9 thru 0xd56c17c (0x5c4 bytes) 2024-09-16T22:16:51.971 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.971 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:51.971 INFO:teuthology.orchestra.run.smithi031.stdout:2527 read 0x8e12d99 thru 0x8e13dd6 (0x103e bytes) 2024-09-16T22:16:51.974 INFO:teuthology.orchestra.run.smithi031.stdout:2528 trunc from 0xe176b41 to 0xafa7f14 2024-09-16T22:16:51.991 INFO:teuthology.orchestra.run.smithi031.stdout:2529 write 0xc40f263 thru 0xc41bf3d (0xccdb bytes) 2024-09-16T22:16:51.995 INFO:teuthology.orchestra.run.smithi031.stdout:2530 writesame 0x7c9dd9 thru 0x7cc724 data_size 0x6(0x294c bytes) 2024-09-16T22:16:51.997 INFO:teuthology.orchestra.run.smithi031.stdout:2531 write 0x1225463 thru 0x123109e (0xbc3c bytes) 2024-09-16T22:16:52.000 INFO:teuthology.orchestra.run.smithi031.stdout:2532 writesame 0x9e2dd42 thru 0x9e390e8 data_size 0x25(0xb3a7 bytes) 2024-09-16T22:16:52.003 INFO:teuthology.orchestra.run.smithi031.stdout:2533 write 0x9949b2a thru 0x994af58 (0x142f bytes) 2024-09-16T22:16:52.004 INFO:teuthology.orchestra.run.smithi031.stdout:2534 write 0x212e697 thru 0x213a676 (0xbfe0 bytes) 2024-09-16T22:16:52.007 INFO:teuthology.orchestra.run.smithi031.stdout:2535 read 0x860b9e5 thru 0x86111a1 (0x57bd bytes) 2024-09-16T22:16:52.008 INFO:teuthology.orchestra.run.smithi031.stdout:2539 write 0x4ff061c thru 0x4ffb49a (0xae7f bytes) 2024-09-16T22:16:52.009 INFO:teuthology.orchestra.run.smithi031.stdout:2540 read 0x2dbc171 thru 0x2dca1da (0xe06a bytes) 2024-09-16T22:16:52.016 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.016 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.021 INFO:teuthology.orchestra.run.smithi031.stdout:2541 trunc from 0xc41bf3e to 0x2035d6d 2024-09-16T22:16:52.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:52.025 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:52.026+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:52.042 INFO:teuthology.orchestra.run.smithi031.stdout:2542 write 0x500cb72 thru 0x501369e (0x6b2d bytes) 2024-09-16T22:16:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:2544 write 0x42769a9 thru 0x427cedc (0x6534 bytes) 2024-09-16T22:16:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:2545 punch from 0x2769f1b to 0x2771dce, (0x7eb3 bytes) 2024-09-16T22:16:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:16:52.054 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:16:52.060 INFO:teuthology.orchestra.run.smithi031.stdout:2547 writesame 0xcd45d68 thru 0xcd541ac data_size 0x9(0xe445 bytes) 2024-09-16T22:16:52.060 INFO:teuthology.orchestra.run.smithi031.stdout:2548 write 0x7cb02f thru 0x7d5c2a (0xabfc bytes) 2024-09-16T22:16:52.062 INFO:teuthology.orchestra.run.smithi031.stdout:2549 write 0xc093ac thru 0xc0d852 (0x44a7 bytes) 2024-09-16T22:16:52.064 INFO:teuthology.orchestra.run.smithi031.stdout:2550 read 0xaf981c3 thru 0xaf9e077 (0x5eb5 bytes) 2024-09-16T22:16:52.064 INFO:teuthology.orchestra.run.smithi031.stdout:2551 read 0x49ba412 thru 0x49baa3b (0x62a bytes) 2024-09-16T22:16:52.067 INFO:teuthology.orchestra.run.smithi031.stdout:2552 trunc from 0xcd541ad to 0xed41e6f 2024-09-16T22:16:52.067 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-16T22:16:51.316063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326349879,"num_objects":1345,"num_object_clones":81,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":992,"num_read":47838,"num_read_kb":173396,"num_write":6452,"num_write_kb":131687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6381,"ondisk_log_size":6381,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":362720,"kb_used_data":83812,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754612000,"statfs":{"total":773094113280,"available":772722688000,"internally_reserved":0,"allocated":85823488,"data_stored":95351024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3496271,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":152,"num_read_kb":285,"num_write":15,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033927"},"pg_stats":[{"pgid":"3.1f","version":"49'224","reported_seq":1751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.566998+0000","last_change":"2024-09-16T22:16:17.136383+0000","last_active":"2024-09-16T22:16:46.566998+0000","last_peered":"2024-09-16T22:16:46.566998+0000","last_clean":"2024-09-16T22:16:46.566998+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:16:46.566998+0000","last_undegraded":"2024-09-16T22:16:46.566998+0000","last_fullsized":"2024-09-16T22:16:46.566998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:37.254425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095346500000000002,"stat_sum":{"num_bytes":11999249,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":34,"num_read":2502,"num_read_kb":5938,"num_write":194,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"47'108","reported_seq":512,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294429+0000","last_change":"2024-09-16T22:16:17.136432+0000","last_active":"2024-09-16T22:16:46.294429+0000","last_peered":"2024-09-16T22:16:46.294429+0000","last_clean":"2024-09-16T22:16:46.294429+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:16:46.294429+0000","last_undegraded":"2024-09-16T22:16:46.294429+0000","last_fullsized":"2024-09-16T22:16:46.294429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:04.096142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078321800000000002,"stat_sum":{"num_bytes":7084633,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":26,"num_read":566,"num_read_kb":12322,"num_write":84,"num_write_kb":30982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"49'146","reported_seq":829,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.568620+0000","last_change":"2024-09-16T22:16:17.128822+0000","last_active":"2024-09-16T22:16:46.568620+0000","last_peered":"2024-09-16T22:16:46.568620+0000","last_clean":"2024-09-16T22:16:46.568620+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:16:46.568620+0000","last_undegraded":"2024-09-16T22:16:46.568620+0000","last_fullsized":"2024-09-16T22:16:46.568620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:04.455146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019949680000000002,"stat_sum":{"num_bytes":6518353,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":19,"num_read":1088,"num_read_kb":1983,"num_write":125,"num_write_kb":982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"48'131","reported_seq":1225,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.467202+0000","last_change":"2024-09-16T22:16:17.131219+0000","last_active":"2024-09-16T22:16:46.467202+0000","last_peered":"2024-09-16T22:16:46.467202+0000","last_clean":"2024-09-16T22:16:46.467202+0000","last_became_active":"2024-09-16T22:16:15.135556+0000","last_became_peered":"2024-09-16T22:16:15.135556+0000","last_unstale":"2024-09-16T22:16:46.467202+0000","last_undegraded":"2024-09-16T22:16:46.467202+0000","last_fullsized":"2024-09-16T22:16:46.467202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:26:53.306823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052703799999999996,"stat_sum":{"num_bytes":4523322,"num_objects":44,"num_object_clones":3,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":33,"num_read":1119,"num_read_kb":4057,"num_write":99,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"49'212","reported_seq":903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.528025+0000","last_change":"2024-09-16T22:16:17.263803+0000","last_active":"2024-09-16T22:16:46.528025+0000","last_peered":"2024-09-16T22:16:46.528025+0000","last_clean":"2024-09-16T22:16:46.528025+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:16:46.528025+0000","last_undegraded":"2024-09-16T22:16:46.528025+0000","last_fullsized":"2024-09-16T22:16:46.528025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:31.454818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053437600000000004,"stat_sum":{"num_bytes":8276329,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":25,"num_read":1710,"num_read_kb":1644,"num_write":183,"num_write_kb":1779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"49'164","reported_seq":5619,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.602300+0000","last_change":"2024-09-16T22:16:17.131812+0000","last_active":"2024-09-16T22:16:46.602300+0000","last_peered":"2024-09-16T22:16:46.602300+0000","last_clean":"2024-09-16T22:16:46.602300+0000","last_became_active":"2024-09-16T22:16:15.126160+0000","last_became_peered":"2024-09-16T22:16:15.126160+0000","last_unstale":"2024-09-16T22:16:46.602300+0000","last_undegraded":"2024-09-16T22:16:46.602300+0000","last_fullsized":"2024-09-16T22:16:46.602300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:52.480013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010623589999999999,"stat_sum":{"num_bytes":15445043,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":5525,"num_read_kb":5598,"num_write":131,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"48'99","reported_seq":384,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.425255+0000","last_change":"2024-09-16T22:16:17.263495+0000","last_active":"2024-09-16T22:16:46.425255+0000","last_peered":"2024-09-16T22:16:46.425255+0000","last_clean":"2024-09-16T22:16:46.425255+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:16:46.425255+0000","last_undegraded":"2024-09-16T22:16:46.425255+0000","last_fullsized":"2024-09-16T22:16:46.425255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:22.625830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040510500000000001,"stat_sum":{"num_bytes":9877981,"num_objects":39,"num_object_clones":2,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":27,"num_read":257,"num_read_kb":898,"num_write":69,"num_write_kb":422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"49'210","reported_seq":314,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294526+0000","last_change":"2024-09-16T22:16:17.136712+0000","last_active":"2024-09-16T22:16:46.294526+0000","last_peered":"2024-09-16T22:16:46.294526+0000","last_clean":"2024-09-16T22:16:46.294526+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:16:46.294526+0000","last_undegraded":"2024-09-16T22:16:46.294526+0000","last_fullsized":"2024-09-16T22:16:46.294526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:52.131127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096381500000000003,"stat_sum":{"num_bytes":4589609,"num_objects":37,"num_object_clones":1,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":31,"num_read":592,"num_read_kb":616,"num_write":252,"num_write_kb":563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"49'388","reported_seq":1079,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294610+0000","last_change":"2024-09-16T22:16:17.135852+0000","last_active":"2024-09-16T22:16:46.294610+0000","last_peered":"2024-09-16T22:16:46.294610+0000","last_clean":"2024-09-16T22:16:46.294610+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:16:46.294610+0000","last_undegraded":"2024-09-16T22:16:46.294610+0000","last_fullsized":"2024-09-16T22:16:46.294610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:30.009725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042473899999999999,"stat_sum":{"num_bytes":15147821,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":29,"num_read":2047,"num_read_kb":8519,"num_write":564,"num_write_kb":17525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"47'137","reported_seq":1407,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276133+0000","last_change":"2024-09-16T22:16:17.127257+0000","last_active":"2024-09-16T22:16:46.276133+0000","last_peered":"2024-09-16T22:16:46.276133+0000","last_clean":"2024-09-16T22:16:46.276133+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:16:46.276133+0000","last_undegraded":"2024-09-16T22:16:46.276133+0000","last_fullsized":"2024-09-16T22:16:46.276133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:13.884897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093962400000000001,"stat_sum":{"num_bytes":13638927,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":1231,"num_read_kb":6032,"num_write":111,"num_write_kb":2031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"49'401","reported_seq":796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418090+0000","last_change":"2024-09-16T22:16:17.135778+0000","last_active":"2024-09-16T22:16:46.418090+0000","last_peered":"2024-09-16T22:16:46.418090+0000","last_clean":"2024-09-16T22:16:46.418090+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:16:46.418090+0000","last_undegraded":"2024-09-16T22:16:46.418090+0000","last_fullsized":"2024-09-16T22:16:46.418090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:36.889739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041336299999999998,"stat_sum":{"num_bytes":12518884,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":1765,"num_read_kb":7416,"num_write":558,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"49'144","reported_seq":448,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560653+0000","last_change":"2024-09-16T22:16:17.128475+0000","last_active":"2024-09-16T22:16:46.560653+0000","last_peered":"2024-09-16T22:16:46.560653+0000","last_clean":"2024-09-16T22:16:46.560653+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:16:46.560653+0000","last_undegraded":"2024-09-16T22:16:46.560653+0000","last_fullsized":"2024-09-16T22:16:46.560653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:42.401536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001932436,"stat_sum":{"num_bytes":9306351,"num_objects":36,"num_object_clones":1,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":28,"num_read":391,"num_read_kb":1711,"num_write":135,"num_write_kb":531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275996+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:16:46.275996+0000","last_peered":"2024-09-16T22:16:46.275996+0000","last_clean":"2024-09-16T22:16:46.275996+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:16:46.275996+0000","last_undegraded":"2024-09-16T22:16:46.275996+0000","last_fullsized":"2024-09-16T22:16:46.275996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"49'422","reported_seq":6951,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.482335+0000","last_change":"2024-09-16T22:16:17.263952+0000","last_active":"2024-09-16T22:16:46.482335+0000","last_peered":"2024-09-16T22:16:46.482335+0000","last_clean":"2024-09-16T22:16:46.482335+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:16:46.482335+0000","last_undegraded":"2024-09-16T22:16:46.482335+0000","last_fullsized":"2024-09-16T22:16:46.482335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:53.349620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063812899999999997,"stat_sum":{"num_bytes":12184410,"num_objects":44,"num_object_clones":2,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":8813,"num_read_kb":13552,"num_write":532,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280213+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:16:46.280213+0000","last_peered":"2024-09-16T22:16:46.280213+0000","last_clean":"2024-09-16T22:16:46.280213+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:16:46.280213+0000","last_undegraded":"2024-09-16T22:16:46.280213+0000","last_fullsized":"2024-09-16T22:16:46.280213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"49'145","reported_seq":633,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.474629+0000","last_change":"2024-09-16T22:16:17.264006+0000","last_active":"2024-09-16T22:16:46.474629+0000","last_peered":"2024-09-16T22:16:46.474629+0000","last_clean":"2024-09-16T22:16:46.474629+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:16:46.474629+0000","last_undegraded":"2024-09-16T22:16:46.474629+0000","last_fullsized":"2024-09-16T22:16:46.474629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:58.355316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065480300000000003,"stat_sum":{"num_bytes":8007175,"num_objects":48,"num_object_clones":1,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":38,"num_read":688,"num_read_kb":4413,"num_write":104,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276002+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:16:46.276002+0000","last_peered":"2024-09-16T22:16:46.276002+0000","last_clean":"2024-09-16T22:16:46.276002+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:16:46.276002+0000","last_undegraded":"2024-09-16T22:16:46.276002+0000","last_fullsized":"2024-09-16T22:16:46.276002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'212","reported_seq":2169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278254+0000","last_change":"2024-09-16T22:16:17.147841+0000","last_active":"2024-09-16T22:16:46.278254+0000","last_peered":"2024-09-16T22:16:46.278254+0000","last_clean":"2024-09-16T22:16:46.278254+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:16:46.278254+0000","last_undegraded":"2024-09-16T22:16:46.278254+0000","last_fullsized":"2024-09-16T22:16:46.278254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:23.882068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053693199999999995,"stat_sum":{"num_bytes":13761474,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":2187,"num_read_kb":6483,"num_write":220,"num_write_kb":3375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.294501+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:16:46.294501+0000","last_peered":"2024-09-16T22:16:46.294501+0000","last_clean":"2024-09-16T22:16:46.294501+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:16:46.294501+0000","last_undegraded":"2024-09-16T22:16:46.294501+0000","last_fullsized":"2024-09-16T22:16:46.294501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"47'135","reported_seq":1395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275868+0000","last_change":"2024-09-16T22:16:17.126856+0000","last_active":"2024-09-16T22:16:46.275868+0000","last_peered":"2024-09-16T22:16:46.275868+0000","last_clean":"2024-09-16T22:16:46.275868+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:16:46.275868+0000","last_undegraded":"2024-09-16T22:16:46.275868+0000","last_fullsized":"2024-09-16T22:16:46.275868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:44.976084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066546900000000002,"stat_sum":{"num_bytes":28609961,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":1453,"num_read_kb":14700,"num_write":98,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"47'208","reported_seq":877,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417895+0000","last_change":"2024-09-16T22:16:17.136232+0000","last_active":"2024-09-16T22:16:46.417895+0000","last_peered":"2024-09-16T22:16:46.417895+0000","last_clean":"2024-09-16T22:16:46.417895+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:16:46.417895+0000","last_undegraded":"2024-09-16T22:16:46.417895+0000","last_fullsized":"2024-09-16T22:16:46.417895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:50.781472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078718899999999999,"stat_sum":{"num_bytes":10561849,"num_objects":51,"num_object_clones":4,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":40,"num_read":1092,"num_read_kb":4610,"num_write":195,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275851+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:16:46.275851+0000","last_peered":"2024-09-16T22:16:46.275851+0000","last_clean":"2024-09-16T22:16:46.275851+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:16:46.275851+0000","last_undegraded":"2024-09-16T22:16:46.275851+0000","last_fullsized":"2024-09-16T22:16:46.275851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"49'126","reported_seq":454,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.437832+0000","last_change":"2024-09-16T22:16:17.136580+0000","last_active":"2024-09-16T22:16:46.437832+0000","last_peered":"2024-09-16T22:16:46.437832+0000","last_clean":"2024-09-16T22:16:46.437832+0000","last_became_active":"2024-09-16T22:16:15.126826+0000","last_became_peered":"2024-09-16T22:16:15.126826+0000","last_unstale":"2024-09-16T22:16:46.437832+0000","last_undegraded":"2024-09-16T22:16:46.437832+0000","last_fullsized":"2024-09-16T22:16:46.437832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:45.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089698499999999999,"stat_sum":{"num_bytes":1989454,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":23,"num_read":675,"num_read_kb":545,"num_write":99,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.337943+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:16:49.337943+0000","last_peered":"2024-09-16T22:16:49.337943+0000","last_clean":"2024-09-16T22:16:49.337943+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:16:49.337943+0000","last_undegraded":"2024-09-16T22:16:49.337943+0000","last_fullsized":"2024-09-16T22:16:49.337943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"49'115","reported_seq":779,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.607632+0000","last_change":"2024-09-16T22:16:46.346779+0000","last_active":"2024-09-16T22:16:46.607632+0000","last_peered":"2024-09-16T22:16:46.607632+0000","last_clean":"2024-09-16T22:16:46.607632+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:16:46.607632+0000","last_undegraded":"2024-09-16T22:16:46.607632+0000","last_fullsized":"2024-09-16T22:16:46.607632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:56.705809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045343400000000001,"stat_sum":{"num_bytes":8221931,"num_objects":38,"num_object_clones":1,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":27,"num_read":571,"num_read_kb":2851,"num_write":101,"num_write_kb":713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275587+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:16:46.275587+0000","last_peered":"2024-09-16T22:16:46.275587+0000","last_clean":"2024-09-16T22:16:46.275587+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:16:46.275587+0000","last_undegraded":"2024-09-16T22:16:46.275587+0000","last_fullsized":"2024-09-16T22:16:46.275587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"47'132","reported_seq":484,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280133+0000","last_change":"2024-09-16T22:16:17.263750+0000","last_active":"2024-09-16T22:16:46.280133+0000","last_peered":"2024-09-16T22:16:46.280133+0000","last_clean":"2024-09-16T22:16:46.280133+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:16:46.280133+0000","last_undegraded":"2024-09-16T22:16:46.280133+0000","last_fullsized":"2024-09-16T22:16:46.280133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:14.235622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060679099999999997,"stat_sum":{"num_bytes":13313472,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":34,"num_read":338,"num_read_kb":14345,"num_write":99,"num_write_kb":10505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275736+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:16:46.275736+0000","last_peered":"2024-09-16T22:16:46.275736+0000","last_clean":"2024-09-16T22:16:46.275736+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:16:46.275736+0000","last_undegraded":"2024-09-16T22:16:46.275736+0000","last_fullsized":"2024-09-16T22:16:46.275736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":164,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276204+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:16:46.276204+0000","last_peered":"2024-09-16T22:16:46.276204+0000","last_clean":"2024-09-16T22:16:46.276204+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:16:46.276204+0000","last_undegraded":"2024-09-16T22:16:46.276204+0000","last_fullsized":"2024-09-16T22:16:46.276204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"49'172","reported_seq":663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.280121+0000","last_change":"2024-09-16T22:16:17.263479+0000","last_active":"2024-09-16T22:16:46.280121+0000","last_peered":"2024-09-16T22:16:46.280121+0000","last_clean":"2024-09-16T22:16:46.280121+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:16:46.280121+0000","last_undegraded":"2024-09-16T22:16:46.280121+0000","last_fullsized":"2024-09-16T22:16:46.280121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:06.292129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450018,"stat_sum":{"num_bytes":4987868,"num_objects":49,"num_object_clones":3,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":35,"num_read":868,"num_read_kb":1843,"num_write":131,"num_write_kb":787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"49'262","reported_seq":814,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.560388+0000","last_change":"2024-09-16T22:16:17.128500+0000","last_active":"2024-09-16T22:16:46.560388+0000","last_peered":"2024-09-16T22:16:46.560388+0000","last_clean":"2024-09-16T22:16:46.560388+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:16:46.560388+0000","last_undegraded":"2024-09-16T22:16:46.560388+0000","last_fullsized":"2024-09-16T22:16:46.560388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:42.030243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001733109,"stat_sum":{"num_bytes":7657856,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":1250,"num_read_kb":3842,"num_write":331,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"49'301","reported_seq":680,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275807+0000","last_change":"2024-09-16T22:16:17.127879+0000","last_active":"2024-09-16T22:16:46.275807+0000","last_peered":"2024-09-16T22:16:46.275807+0000","last_clean":"2024-09-16T22:16:46.275807+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:16:46.275807+0000","last_undegraded":"2024-09-16T22:16:46.275807+0000","last_fullsized":"2024-09-16T22:16:46.275807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:53.972422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001383212,"stat_sum":{"num_bytes":9549839,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":959,"num_read_kb":4045,"num_write":347,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"52'206","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T22:16:49.357011+0000","last_change":"2024-09-16T22:16:17.263197+0000","last_active":"2024-09-16T22:16:49.357011+0000","last_peered":"2024-09-16T22:16:49.357011+0000","last_clean":"2024-09-16T22:16:49.357011+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:16:49.357011+0000","last_undegraded":"2024-09-16T22:16:49.357011+0000","last_fullsized":"2024-09-16T22:16:49.357011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:52:46.225583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476284,"stat_sum":{"num_bytes":16326061,"num_objects":58,"num_object_clones":4,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":38,"num_read":1422,"num_read_kb":7919,"num_write":178,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"47'132","reported_seq":606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.417902+0000","last_change":"2024-09-16T22:16:17.135988+0000","last_active":"2024-09-16T22:16:46.417902+0000","last_peered":"2024-09-16T22:16:46.417902+0000","last_clean":"2024-09-16T22:16:46.417902+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:16:46.417902+0000","last_undegraded":"2024-09-16T22:16:46.417902+0000","last_fullsized":"2024-09-16T22:16:46.417902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:22.321858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062643099999999999,"stat_sum":{"num_bytes":19277734,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":443,"num_read_kb":4970,"num_write":92,"num_write_kb":2578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"47'204","reported_seq":854,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.480721+0000","last_change":"2024-09-16T22:16:17.128860+0000","last_active":"2024-09-16T22:16:46.480721+0000","last_peered":"2024-09-16T22:16:46.480721+0000","last_clean":"2024-09-16T22:16:46.480721+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:16:46.480721+0000","last_undegraded":"2024-09-16T22:16:46.480721+0000","last_fullsized":"2024-09-16T22:16:46.480721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:43.259469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020238500000000002,"stat_sum":{"num_bytes":1915593,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":29,"num_read":1553,"num_read_kb":1537,"num_write":176,"num_write_kb":4702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'136","reported_seq":672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276233+0000","last_change":"2024-09-16T22:16:17.127452+0000","last_active":"2024-09-16T22:16:46.276233+0000","last_peered":"2024-09-16T22:16:46.276233+0000","last_clean":"2024-09-16T22:16:46.276233+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:16:46.276233+0000","last_undegraded":"2024-09-16T22:16:46.276233+0000","last_fullsized":"2024-09-16T22:16:46.276233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:43.537885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090449699999999998,"stat_sum":{"num_bytes":11838195,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":27,"num_read":504,"num_read_kb":13038,"num_write":110,"num_write_kb":15147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"47'126","reported_seq":460,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.418041+0000","last_change":"2024-09-16T22:16:17.135885+0000","last_active":"2024-09-16T22:16:46.418041+0000","last_peered":"2024-09-16T22:16:46.418041+0000","last_clean":"2024-09-16T22:16:46.418041+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:16:46.418041+0000","last_undegraded":"2024-09-16T22:16:46.418041+0000","last_fullsized":"2024-09-16T22:16:46.418041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.071537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050898299999999996,"stat_sum":{"num_bytes":3699813,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":35,"num_read":508,"num_read_kb":445,"num_write":89,"num_write_kb":363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"47'161","reported_seq":502,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.276141+0000","last_change":"2024-09-16T22:16:17.131207+0000","last_active":"2024-09-16T22:16:46.276141+0000","last_peered":"2024-09-16T22:16:46.276141+0000","last_clean":"2024-09-16T22:16:46.276141+0000","last_became_active":"2024-09-16T22:16:15.127331+0000","last_became_peered":"2024-09-16T22:16:15.127331+0000","last_unstale":"2024-09-16T22:16:46.276141+0000","last_undegraded":"2024-09-16T22:16:46.276141+0000","last_fullsized":"2024-09-16T22:16:46.276141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:17.655441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055399799999999999,"stat_sum":{"num_bytes":4041720,"num_objects":39,"num_object_clones":3,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":32,"num_read":605,"num_read_kb":2847,"num_write":169,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"47'422","reported_seq":1669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.275589+0000","last_change":"2024-09-16T22:16:17.126205+0000","last_active":"2024-09-16T22:16:46.275589+0000","last_peered":"2024-09-16T22:16:46.275589+0000","last_clean":"2024-09-16T22:16:46.275589+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:16:46.275589+0000","last_undegraded":"2024-09-16T22:16:46.275589+0000","last_fullsized":"2024-09-16T22:16:46.275589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:02.069887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049330399999999996,"stat_sum":{"num_bytes":14962163,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":36,"num_read":3070,"num_read_kb":12486,"num_write":513,"num_write_kb":3434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"48'160","reported_seq":479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.278240+0000","last_change":"2024-09-16T22:16:17.147648+0000","last_active":"2024-09-16T22:16:46.278240+0000","last_peered":"2024-09-16T22:16:46.278240+0000","last_clean":"2024-09-16T22:16:46.278240+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:16:46.278240+0000","last_undegraded":"2024-09-16T22:16:46.278240+0000","last_fullsized":"2024-09-16T22:16:46.278240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:55.631170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044714499999999998,"stat_sum":{"num_bytes":11391311,"num_objects":45,"num_object_clones":2,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":38,"num_read":675,"num_read_kb":936,"num_write":128,"num_write_kb":2582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"47'165","reported_seq":665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T22:16:46.464753+0000","last_change":"2024-09-16T22:16:17.127621+0000","last_active":"2024-09-16T22:16:46.464753+0000","last_peered":"2024-09-16T22:16:46.464753+0000","last_clean":"2024-09-16T22:16:46.464753+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:16:46.464753+0000","last_undegraded":"2024-09-16T22:16:46.464753+0000","last_fullsized":"2024-09-16T22:16:46.464753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:32.968348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682444,"stat_sum":{"num_bytes":4535111,"num_objects":41,"num_object_clones":1,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":1305,"num_read_kb":1200,"num_write":132,"num_write_kb":4014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":325759492,"num_objects":1341,"num_object_clones":81,"num_object_copies":2682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1341,"num_whiteouts":992,"num_read":47772,"num_read_kb":173341,"num_write":6349,"num_write_kb":130057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":80510976,"data_stored":92499477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6306,"ondisk_log_size":6306,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44096,"kb_used_data":8748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327744,"statfs":{"total":96636764160,"available":96591609856,"internally_reserved":0,"allocated":8957952,"data_stored":12264716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42856,"kb_used_data":7500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328984,"statfs":{"total":96636764160,"available":96592879616,"internally_reserved":0,"allocated":7680000,"data_stored":7234675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49476,"kb_used_data":14124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322364,"statfs":{"total":96636764160,"available":96586100736,"internally_reserved":0,"allocated":14462976,"data_stored":15406271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41696,"kb_used_data":6348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330144,"statfs":{"total":96636764160,"available":96594067456,"internally_reserved":0,"allocated":6500352,"data_stored":7396383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":24280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":24862720,"data_stored":27700733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39040,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332800,"statfs":{"total":96636764160,"available":96596787200,"internally_reserved":0,"allocated":3772416,"data_stored":3296049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43260,"kb_used_data":7912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328580,"statfs":{"total":96636764160,"available":96592465920,"internally_reserved":0,"allocated":8101888,"data_stored":8130735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42408,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329432,"statfs":{"total":96636764160,"available":96593338368,"internally_reserved":0,"allocated":11485184,"data_stored":13921462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7606272,"data_stored":7938493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":2472825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24494080,"data_stored":27612864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6049792,"data_stored":7246016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13754368,"data_stored":15028696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":6411451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8417280,"data_stored":12032109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11005952,"data_stored":13757023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:16:52.069 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [5, 0] 2024-09-16T22:16:52.069 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.0', '5', '0'] 2024-09-16T22:16:52.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.0 5 0 2024-09-16T22:16:52.073 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.073 INFO:teuthology.orchestra.run.smithi031.stdout:2554 punch from 0xe9b14f1 to 0xe9b5b84, (0x4693 bytes) 2024-09-16T22:16:52.073 INFO:teuthology.orchestra.run.smithi031.stdout:2555 read 0xaab490c thru 0xaac1b2f (0xd224 bytes) 2024-09-16T22:16:52.073 INFO:teuthology.orchestra.run.smithi031.stdout:2556 read 0xae8cd35 thru 0xae94708 (0x79d4 bytes) 2024-09-16T22:16:52.073 INFO:teuthology.orchestra.run.smithi031.stdout:2557 read 0xb62992d thru 0xb62f664 (0x5d38 bytes) 2024-09-16T22:16:52.082 INFO:teuthology.orchestra.run.smithi031.stdout:2558 trunc from 0xed41e6f to 0x5e7b500 2024-09-16T22:16:52.090 INFO:teuthology.orchestra.run.smithi031.stdout:2559 read 0x3e23df5 thru 0x3e31b2b (0xdd37 bytes) 2024-09-16T22:16:52.090 INFO:teuthology.orchestra.run.smithi031.stdout:2560 read 0x293140d thru 0x2939131 (0x7d25 bytes) 2024-09-16T22:16:52.094 INFO:teuthology.orchestra.run.smithi031.stdout:2561 trunc from 0x5e7b500 to 0x84c4e1f 2024-09-16T22:16:52.099 INFO:teuthology.orchestra.run.smithi031.stdout:2563 read 0x103f0d9 thru 0x104164a (0x2572 bytes) 2024-09-16T22:16:52.100 INFO:teuthology.orchestra.run.smithi031.stdout:2565 read 0x7de76ee thru 0x7df1380 (0x9c93 bytes) 2024-09-16T22:16:52.100 INFO:teuthology.orchestra.run.smithi031.stdout:2566 read 0x4213eea thru 0x421d219 (0x9330 bytes) 2024-09-16T22:16:52.101 INFO:teuthology.orchestra.run.smithi031.stdout:2567 read 0x60fad6a thru 0x610a844 (0xfadb bytes) 2024-09-16T22:16:52.101 INFO:teuthology.orchestra.run.smithi031.stdout:2568 writesame 0x192dc48 thru 0x193bcd0 data_size 0x1(0xe089 bytes) 2024-09-16T22:16:52.125 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:52.127+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:52.130 INFO:teuthology.orchestra.run.smithi031.stdout:2569 read 0x3ecf1f5 thru 0x3ed953e (0xa34a bytes) 2024-09-16T22:16:52.143 INFO:teuthology.orchestra.run.smithi031.stdout:2570 write 0xe974711 thru 0xe9782d8 (0x3bc8 bytes) 2024-09-16T22:16:52.147 INFO:teuthology.orchestra.run.smithi031.stdout:2572 write 0x13540d thru 0x142472 (0xd066 bytes) 2024-09-16T22:16:52.149 INFO:teuthology.orchestra.run.smithi031.stdout:2573 writesame 0x4baf424 thru 0x4bbc7c9 data_size 0x1(0xd3a6 bytes) 2024-09-16T22:16:52.173 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.173 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.176 INFO:teuthology.orchestra.run.smithi031.stdout:2574 writesame 0x1440bb3 thru 0x144f7e6 data_size 0x6(0xec34 bytes) 2024-09-16T22:16:52.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:52.183 INFO:teuthology.orchestra.run.smithi031.stdout:2577 read 0x8afe7f7 thru 0x8b0bcec (0xd4f6 bytes) 2024-09-16T22:16:52.183 INFO:teuthology.orchestra.run.smithi031.stdout:2579 punch from 0xd0a0a78 to 0xd0ab679, (0xac01 bytes) 2024-09-16T22:16:52.183 INFO:teuthology.orchestra.run.smithi031.stdout:2580 write 0x45eb5fd thru 0x45f365e (0x8062 bytes) 2024-09-16T22:16:52.184 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.197 INFO:teuthology.orchestra.run.smithi031.stdout:2584 trunc from 0xe9782d9 to 0x3a11417 2024-09-16T22:16:52.208 INFO:teuthology.orchestra.run.smithi031.stdout:2585 write 0xd738bb thru 0xd81498 (0xdbde bytes) 2024-09-16T22:16:52.219 INFO:teuthology.orchestra.run.smithi031.stdout:2586 trunc from 0x3a11417 to 0xb813c85 2024-09-16T22:16:52.225 INFO:teuthology.orchestra.run.smithi031.stdout:2587 write 0x760f7fd thru 0x761f30f (0xfb13 bytes) 2024-09-16T22:16:52.225 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:52.227+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:52.227 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.236 INFO:teuthology.orchestra.run.smithi031.stdout:2591 writesame 0xe705534 thru 0xe70d027 data_size 0x3d(0x7af4 bytes) 2024-09-16T22:16:52.236 INFO:teuthology.orchestra.run.smithi031.stdout:2593 read 0x3c63eb1 thru 0x3c6d911 (0x9a61 bytes) 2024-09-16T22:16:52.237 INFO:teuthology.orchestra.run.smithi031.stdout:2594 punch from 0xb5efc19 to 0xb5fda13, (0xddfa bytes) 2024-09-16T22:16:52.248 INFO:teuthology.orchestra.run.smithi031.stdout:2595 trunc from 0xe70d028 to 0x458cc3b 2024-09-16T22:16:52.259 INFO:teuthology.orchestra.run.smithi031.stdout:2596 read 0x32c2b1e thru 0x32cc330 (0x9813 bytes) 2024-09-16T22:16:52.270 INFO:teuthology.orchestra.run.smithi031.stdout:2597 write 0x877309c thru 0x87820a7 (0xf00c bytes) 2024-09-16T22:16:52.273 INFO:teuthology.orchestra.run.smithi031.stdout:2598 write 0x5e8451b thru 0x5e91669 (0xd14f bytes) 2024-09-16T22:16:52.274 INFO:teuthology.orchestra.run.smithi031.stdout:2599 write 0x8723dde thru 0x87330db (0xf2fe bytes) 2024-09-16T22:16:52.283 INFO:teuthology.orchestra.run.smithi031.stdout:2600 write 0xa6cf427 thru 0xa6df39d (0xff77 bytes) 2024-09-16T22:16:52.286 INFO:teuthology.orchestra.run.smithi031.stdout:2601 write 0x151110e thru 0x1512665 (0x1558 bytes) 2024-09-16T22:16:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:2603 writesame 0xa27758f thru 0xa280e39 data_size 0x79(0x98ab bytes) 2024-09-16T22:16:52.289 INFO:teuthology.orchestra.run.smithi031.stdout:2604 write 0x8084c44 thru 0x808c4b9 (0x7876 bytes) 2024-09-16T22:16:52.290 INFO:teuthology.orchestra.run.smithi031.stdout:2605 read 0x6d0f08a thru 0x6d10f08 (0x1e7f bytes) 2024-09-16T22:16:52.298 INFO:teuthology.orchestra.run.smithi031.stdout:2606 write 0xd0d3c49 thru 0xd0db981 (0x7d39 bytes) 2024-09-16T22:16:52.301 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.301 INFO:teuthology.orchestra.run.smithi031.stdout:2608 read 0x671162f thru 0x671f2b6 (0xdc88 bytes) 2024-09-16T22:16:52.301 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.302 INFO:teuthology.orchestra.run.smithi031.stdout:2610 write 0x218711b thru 0x2191911 (0xa7f7 bytes) 2024-09-16T22:16:52.303 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.303 INFO:teuthology.orchestra.run.smithi031.stdout:2613 writesame 0x926fdfa thru 0x927815f data_size 0x16(0x8366 bytes) 2024-09-16T22:16:52.306 INFO:teuthology.orchestra.run.smithi031.stdout:2615 read 0x344690e thru 0x344ffb7 (0x96aa bytes) 2024-09-16T22:16:52.307 INFO:teuthology.orchestra.run.smithi031.stdout:2617 trunc from 0xd0db982 to 0xc61429a 2024-09-16T22:16:52.313 INFO:teuthology.orchestra.run.smithi031.stdout:2618 write 0x45aabb2 thru 0x45ab308 (0x757 bytes) 2024-09-16T22:16:52.315 INFO:teuthology.orchestra.run.smithi031.stdout:2619 read 0x51a1117 thru 0x51a2466 (0x1350 bytes) 2024-09-16T22:16:52.316 INFO:teuthology.orchestra.run.smithi031.stdout:2620 read 0xb4b9dcd thru 0xb4ba736 (0x96a bytes) 2024-09-16T22:16:52.316 INFO:teuthology.orchestra.run.smithi031.stdout:2621 read 0x292a783 thru 0x2936e9a (0xc718 bytes) 2024-09-16T22:16:52.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.316 INFO:teuthology.orchestra.run.smithi031.stdout:2623 writesame 0x214f101 thru 0x215082e data_size 0x6(0x172e bytes) 2024-09-16T22:16:52.317 INFO:teuthology.orchestra.run.smithi031.stdout:2624 punch from 0xa87ffa9 to 0xa882637, (0x268e bytes) 2024-09-16T22:16:52.317 INFO:teuthology.orchestra.run.smithi031.stdout:2625 write 0x8b32895 thru 0x8b3600f (0x377b bytes) 2024-09-16T22:16:52.320 INFO:teuthology.orchestra.run.smithi031.stdout:2626 read 0x4fd9a68 thru 0x4fdef37 (0x54d0 bytes) 2024-09-16T22:16:52.321 INFO:teuthology.orchestra.run.smithi031.stdout:2627 read 0x15c7e87 thru 0x15d51e1 (0xd35b bytes) 2024-09-16T22:16:52.321 INFO:teuthology.orchestra.run.smithi031.stdout:2628 read 0x7ab23dc thru 0x7abebe6 (0xc80b bytes) 2024-09-16T22:16:52.321 INFO:teuthology.orchestra.run.smithi031.stdout:2629 read 0x600438c thru 0x6007282 (0x2ef7 bytes) 2024-09-16T22:16:52.321 INFO:teuthology.orchestra.run.smithi031.stdout:2630 write 0xa762a40 thru 0xa76e1b9 (0xb77a bytes) 2024-09-16T22:16:52.325 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:52.327+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:52.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:16:52.328 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.328 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.334 INFO:teuthology.orchestra.run.smithi031.stdout:2632 trunc from 0xc61429a to 0x2efbda7 2024-09-16T22:16:52.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:52.348 INFO:teuthology.orchestra.run.smithi031.stdout:2634 read 0xa1408e thru 0xa1ffb3 (0xbf26 bytes) 2024-09-16T22:16:52.348 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.365 INFO:teuthology.orchestra.run.smithi031.stdout:2638 writesame 0xc98b399 thru 0xc98db74 data_size 0x2(0x27dc bytes) 2024-09-16T22:16:52.365 INFO:teuthology.orchestra.run.smithi031.stdout:2639 write 0xb3d0615 thru 0xb3d3501 (0x2eed bytes) 2024-09-16T22:16:52.368 INFO:teuthology.orchestra.run.smithi031.stdout:2640 read 0xb1a5b51 thru 0xb1b55aa (0xfa5a bytes) 2024-09-16T22:16:52.369 INFO:teuthology.orchestra.run.smithi031.stdout:2641 punch from 0xa572f0d to 0xa57429c, (0x138f bytes) 2024-09-16T22:16:52.369 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.369 INFO:teuthology.orchestra.run.smithi031.stdout:2643 punch from 0x2a5e8a6 to 0x2a653e1, (0x6b3b bytes) 2024-09-16T22:16:52.369 INFO:teuthology.orchestra.run.smithi031.stdout:2644 writesame 0x478aa94 thru 0x4796d73 data_size 0x10(0xc2e0 bytes) 2024-09-16T22:16:52.384 INFO:teuthology.orchestra.run.smithi031.stdout:2646 trunc from 0xc98db75 to 0x12756a3 2024-09-16T22:16:52.420 INFO:teuthology.orchestra.run.smithi031.stdout:2647 write 0xc7a6cf8 thru 0xc7b1ca9 (0xafb2 bytes) 2024-09-16T22:16:52.426 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:52.428+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:52.428 INFO:teuthology.orchestra.run.smithi031.stdout:2649 write 0xcb1da3e thru 0xcb23a68 (0x602b bytes) 2024-09-16T22:16:52.439 INFO:teuthology.orchestra.run.smithi031.stdout:2651 write 0xe651d41 thru 0xe65b74b (0x9a0b bytes) 2024-09-16T22:16:52.443 INFO:teuthology.orchestra.run.smithi031.stdout:2654 punch from 0xcbebc to 0xd3a20, (0x7b64 bytes) 2024-09-16T22:16:52.443 INFO:teuthology.orchestra.run.smithi031.stdout:2655 read 0x905fa71 thru 0x906800d (0x859d bytes) 2024-09-16T22:16:52.443 INFO:teuthology.orchestra.run.smithi031.stdout:2656 read 0x81509fa thru 0x8155273 (0x487a bytes) 2024-09-16T22:16:52.443 INFO:teuthology.orchestra.run.smithi031.stdout:2658 write 0x3c373c0 thru 0x3c38c48 (0x1889 bytes) 2024-09-16T22:16:52.444 INFO:teuthology.orchestra.run.smithi031.stdout:2659 write 0x3daa882 thru 0x3db774d (0xcecc bytes) 2024-09-16T22:16:52.445 INFO:teuthology.orchestra.run.smithi031.stdout:2660 trunc from 0xe65b74c to 0xe405468 2024-09-16T22:16:52.453 INFO:teuthology.orchestra.run.smithi031.stdout:2661 writesame 0x89705cc thru 0x8979abb data_size 0x8(0x94f0 bytes) 2024-09-16T22:16:52.458 INFO:teuthology.orchestra.run.smithi031.stdout:2663 read 0x231d270 thru 0x2328d2d (0xbabe bytes) 2024-09-16T22:16:52.458 INFO:teuthology.orchestra.run.smithi031.stdout:2668 write 0x6dbc17d thru 0x6dc9aad (0xd931 bytes) 2024-09-16T22:16:52.462 INFO:teuthology.orchestra.run.smithi031.stdout:2670 punch from 0x3b004a to 0x3bce8a, (0xce40 bytes) 2024-09-16T22:16:52.462 INFO:teuthology.orchestra.run.smithi031.stdout:2671 write 0x7177a4b thru 0x717c54c (0x4b02 bytes) 2024-09-16T22:16:52.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.465 INFO:teuthology.orchestra.run.smithi031.stdout:2674 punch from 0xa710e4a to 0xa71b291, (0xa447 bytes) 2024-09-16T22:16:52.465 INFO:teuthology.orchestra.run.smithi031.stdout:2675 punch from 0x111e952 to 0x112c628, (0xdcd6 bytes) 2024-09-16T22:16:52.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.465 INFO:teuthology.orchestra.run.smithi031.stdout:2677 writesame 0x8d62e2e thru 0x8d6401c data_size 0x1(0x11ef bytes) 2024-09-16T22:16:52.468 INFO:teuthology.orchestra.run.smithi031.stdout:2679 read 0xf8aec8 thru 0xf9768a (0xc7c3 bytes) 2024-09-16T22:16:52.468 INFO:teuthology.orchestra.run.smithi031.stdout:2680 punch from 0xa1992c4 to 0xa1a7b80, (0xe8bc bytes) 2024-09-16T22:16:52.468 INFO:teuthology.orchestra.run.smithi031.stdout:2681 writesame 0x89a9c6 thru 0x8a711b data_size 0xa2(0xc756 bytes) 2024-09-16T22:16:52.470 INFO:teuthology.orchestra.run.smithi031.stdout:2682 write 0x6c29b37 thru 0x6c34698 (0xab62 bytes) 2024-09-16T22:16:52.471 INFO:teuthology.orchestra.run.smithi031.stdout:2683 read 0x1d31e23 thru 0x1d3440e (0x25ec bytes) 2024-09-16T22:16:52.472 INFO:teuthology.orchestra.run.smithi031.stdout:2685 read 0x186341f thru 0x186c44b (0x902d bytes) 2024-09-16T22:16:52.472 INFO:teuthology.orchestra.run.smithi031.stdout:2686 read 0x676837c thru 0x676bdae (0x3a33 bytes) 2024-09-16T22:16:52.472 INFO:teuthology.orchestra.run.smithi031.stdout:2687 write 0xfba6f0 thru 0xfc9ebd (0xf7ce bytes) 2024-09-16T22:16:52.473 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.473 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:52.474 INFO:teuthology.orchestra.run.smithi031.stdout:2691 write 0x370c63c thru 0x3716879 (0xa23e bytes) 2024-09-16T22:16:52.475 INFO:teuthology.orchestra.run.smithi031.stdout:2692 punch from 0xd703e51 to 0xd70a042, (0x61f1 bytes) 2024-09-16T22:16:52.475 INFO:teuthology.orchestra.run.smithi031.stdout:2694 read 0xbf8877 thru 0xbfb98a (0x3114 bytes) 2024-09-16T22:16:52.485 INFO:teuthology.orchestra.run.smithi031.stdout:2696 trunc from 0xe405468 to 0x51d8291 2024-09-16T22:16:52.486 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.486 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:2697 punch from 0x276714f to 0x2767d44, (0xbf5 bytes) 2024-09-16T22:16:52.515 INFO:teuthology.orchestra.run.smithi031.stdout:2698 trunc from 0x51d8291 to 0xa3127ca 2024-09-16T22:16:52.522 INFO:teuthology.orchestra.run.smithi031.stdout:2700 trunc from 0xa3127ca to 0x986e800 2024-09-16T22:16:52.527 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:52.528+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:52.528 INFO:teuthology.orchestra.run.smithi031.stdout:2701 write 0x4ece12d thru 0x4ece41a (0x2ee bytes) 2024-09-16T22:16:52.531 INFO:teuthology.orchestra.run.smithi031.stdout:2702 clone 24 order 18 su 4096 sc 4 2024-09-16T22:16:52.608 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.609 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:52.658 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:52.660+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:52.729 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:52.731+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:52.764 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.764 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:52.857 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:52.859+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:52.921 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:52.921 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:52.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:52.931 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:52.933+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:53.033 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:53.035+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:53.078 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:53.078 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:53.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:53.134 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:53.136+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:53.193 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:53.193 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:53.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:53.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:53.243 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:53.245+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:53.314 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.0 pg_upmap mapping to [5,0] 2024-09-16T22:16:53.323 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:16:53.346 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:53.347+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:53.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:53.380 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:53.397 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:53.414 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:53.414 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:53.437 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:53.438+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:53.446 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:53.447 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:53.464 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:53.464 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:53.482 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:53.499 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:53.499 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:53.500 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:53.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:53.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:53.517 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:53.537 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:53.539+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:53.624 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:53.624 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:53.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:53.637 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:53.639+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:53.738 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:53.739+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1659308789999998, 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:16:53.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:16:53.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:16:53.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1659115390000001, 2024-09-16T22:16:53.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:16:53.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:16:53.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1655071389999998, 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:16:53.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:16:53.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45048 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:16:53.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.163833694, 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676787000000002, 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45048 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579108+0000", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579110+0000", 2024-09-16T22:16:53.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579116+0000", 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579120+0000", 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630608+0000", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051487796000000002 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630687+0000", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8709999999999997e-05 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630784+0000", 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7190999999999998e-05 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1638118689999999, 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:16:53.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:16:53.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.163781336, 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1593804369999998, 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:16:53.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1593599559999999, 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.15933938, 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1593148950000001, 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1592933240000001, 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:16:53.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45147 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1532355499999998, 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051674649000000003, 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45147 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589706+0000", 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589708+0000", 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9599999999999999e-06 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589714+0000", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589720+0000", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.383e-06 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641214+0000", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051494686999999997 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641285+0000", 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0795000000000002e-05 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641380+0000", 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5118999999999995e-05 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1532147259999999, 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:16:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:16:53.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1531943939999998, 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:16:53.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:16:53.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:16:53.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1531731750000001, 2024-09-16T22:16:53.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1531518030000001, 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:16:53.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1531309649999999, 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1531103840000001, 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:16:53.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.153088817, 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:53.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1484419429999999, 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:53.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:53.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:53.837 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:53.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:53.839 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:53.840+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:53.940 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:53.941+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:53.987 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:16:53.987 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:53.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:54.040 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:54.042+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:54.103 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:54.103 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:54.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:54.141 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:54.143+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:54.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.796137928, 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:54.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:54.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31471 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.727011247, 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020335977000000002, 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31471 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496617+0000", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496624+0000", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8589999999999999e-06 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496642+0000", 2024-09-16T22:16:54.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142e-05 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496648+0000", 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516812+0000", 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020163871 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516873+0000", 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0956999999999999e-05 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516953+0000", 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9983999999999999e-05 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31473 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:54.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726970958000001, 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020442202, 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31473 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:54.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496657+0000", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496658+0000", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496661+0000", 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.96e-06 2024-09-16T22:16:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496668+0000", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3259999999999998e-06 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496673+0000", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4040000000000003e-06 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.516979+0000", 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020305882000000001 2024-09-16T22:16:54.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517041+0000", 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1307000000000001e-05 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517099+0000", 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8817000000000003e-05 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31475 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726948302, 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021013756000000001, 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31475 2024-09-16T22:16:54.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496680+0000", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496682+0000", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496689+0000", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.652e-06 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496694+0000", 2024-09-16T22:16:54.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517567+0000", 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020872472 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517631+0000", 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3998000000000006e-05 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517694+0000", 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3106000000000002e-05 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31567 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726866365999999, 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02048005, 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31567 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496762+0000", 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496765+0000", 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.219e-06 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496772+0000", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496777+0000", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8790000000000002e-06 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517122+0000", 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020345268999999999 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517182+0000", 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0195000000000002e-05 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517242+0000", 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9512999999999998e-05 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31568 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726844863, 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020607819999999999, 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31568 2024-09-16T22:16:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496783+0000", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496784+0000", 2024-09-16T22:16:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496786+0000", 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496801+0000", 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5517000000000001e-05 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496807+0000", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7060000000000001e-06 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517265+0000", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020458418999999999 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517329+0000", 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3967999999999995e-05 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517391+0000", 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1914000000000004e-05 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31569 3.9 3:961cdb63:::rbd_data.11cd7b8f383d.0000000000000112:e [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726813278, 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020725204000000001, 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31569 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496815+0000", 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496818+0000", 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.464e-06 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496826+0000", 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8159999999999994e-06 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496831+0000", 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489999999999996e-06 2024-09-16T22:16:54.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517414+0000", 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020582975999999999 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517480+0000", 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5260999999999997e-05 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517540+0000", 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0537e-05 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31575 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726789066, 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021677216999999999, 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31575 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496839+0000", 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496840+0000", 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496842+0000", 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5110000000000002e-06 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496849+0000", 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496854+0000", 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517723+0000", 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020868653000000001 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517796+0000", 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399000000000007e-05 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518516+0000", 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072027500000000004 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31576 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726767079, 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021185559999999999, 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31576 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496861+0000", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496862+0000", 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496864+0000", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3549999999999999e-06 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496870+0000", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119999999999998e-06 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496875+0000", 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9320000000000004e-06 2024-09-16T22:16:54.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517895+0000", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021019867000000001 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.517972+0000", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6711000000000005e-05 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518047+0000", 2024-09-16T22:16:54.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.483e-05 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31585 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726746339, 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021312514000000001, 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31585 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496882+0000", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496884+0000", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496890+0000", 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1829999999999999e-06 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496896+0000", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0660000000000003e-06 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518071+0000", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021175678 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518134+0000", 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3263999999999996e-05 2024-09-16T22:16:54.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518194+0000", 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9871999999999999e-05 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31586 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726725369, 2024-09-16T22:16:54.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021438353, 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31586 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:54.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496903+0000", 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496912+0000", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5879999999999998e-06 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496925+0000", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3648999999999999e-05 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496931+0000", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518219+0000", 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021288517 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518279+0000", 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0145000000000001e-05 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518341+0000", 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1717000000000006e-05 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31593 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726688937, 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021611253, 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31593 2024-09-16T22:16:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496939+0000", 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496940+0000", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496943+0000", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1109999999999999e-06 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496949+0000", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4699999999999999e-06 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496954+0000", 2024-09-16T22:16:54.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000001e-06 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518371+0000", 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021416918 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518442+0000", 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1662000000000006e-05 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518550+0000", 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108034 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31594 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726667552, 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021755911999999999, 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31594 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496961+0000", 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496963+0000", 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496969+0000", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496975+0000", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.288e-06 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518576+0000", 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021600891000000001 2024-09-16T22:16:54.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518640+0000", 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3905000000000006e-05 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518717+0000", 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31595 3.11 3:893eddce:::rbd_data.11cd7b8f383d.00000000000000f3:e [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726646231, 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021907319000000001, 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31595 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496982+0000", 2024-09-16T22:16:54.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496985+0000", 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8849999999999999e-06 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496991+0000", 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2759999999999997e-06 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.496997+0000", 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518748+0000", 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021751047999999999 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518811+0000", 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518889+0000", 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8641999999999997e-05 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31603 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726624961000001, 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022076842999999999, 2024-09-16T22:16:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31603 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497003+0000", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497004+0000", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497006+0000", 2024-09-16T22:16:54.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0549999999999998e-06 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497011+0000", 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5860000000000004e-06 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497016+0000", 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4800000000000003e-06 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.518950+0000", 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021934694000000001 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519017+0000", 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6538000000000006e-05 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519080+0000", 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31604 3.11 3:89bbb1ca:::rbd_data.11cd7b8f383d.00000000000000fb:e [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.72660634, 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022218702, 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31604 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:54.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497022+0000", 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497025+0000", 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3259999999999998e-06 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497031+0000", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8370000000000002e-06 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497035+0000", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519107+0000", 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022072692000000001 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519172+0000", 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4672999999999996e-05 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519241+0000", 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8367000000000005e-05 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31605 3.11 3.538ddd91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726586832000001, 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022265193999999999, 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31605 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497041+0000", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497042+0000", 2024-09-16T22:16:54.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497052+0000", 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0073e-05 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497059+0000", 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.007e-06 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497064+0000", 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6559999999999997e-06 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519299+0000", 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022234296000000001 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519307+0000", 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7500000000000003e-06 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31627 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726320411, 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021960872999999999, 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31627 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497308+0000", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497310+0000", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9980000000000002e-06 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497316+0000", 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497321+0000", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6909999999999996e-06 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519262+0000", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021940867999999999 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519269+0000", 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6590000000000001e-06 2024-09-16T22:16:54.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:31628 3.9 3.c6db3869 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.726278756999999, 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021947498999999999, 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:54.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31628 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497349+0000", 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497350+0000", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497352+0000", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4679999999999999e-06 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497359+0000", 2024-09-16T22:16:54.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.497365+0000", 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.327e-06 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519290+0000", 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021925153999999999 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:39.519297+0000", 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1489999999999997e-06 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:42589 3.16 e51/23 3:6bec85e1:::rbd_data.11cd49ca855d.0000000000000003:head v 51'165, mlcod=51'162)", 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0578489219999998, 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028158426, 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:54.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165779+0000", 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:54.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.165781+0000", 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7910000000000001e-06 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169904+0000", 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041225469999999998 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169940+0000", 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6180000000000003e-05 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.169955+0000", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5707000000000001e-05 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.170045+0000", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0002999999999999e-05 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.170076+0000", 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0037999999999999e-05 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.193881+0000", 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023805702000000002 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.193938+0000", 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6458000000000001e-05 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:54.312 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:54.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:54.314 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:54.315+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:54.415 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:54.416+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:54.432 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:54.432 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:54.432 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:54.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:54.516 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:54.517+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:54.545 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:54.546 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:54.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:54.639 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:54.641+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:54.699 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31904 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 925696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511642+0000", 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188926015, 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026792046, 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31904 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511642+0000", 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511642+0000", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511642+0000", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511644+0000", 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.646e-06 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511650+0000", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7010000000000001e-06 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511654+0000", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5410000000000002e-06 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538361+0000", 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02670703 2024-09-16T22:16:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538397+0000", 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6091999999999999e-05 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538434+0000", 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.667e-05 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31906 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1056768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511660+0000", 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188908291000001, 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026866414000000002, 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31906 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511660+0000", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511660+0000", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511660+0000", 2024-09-16T22:16:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3599999999999999e-07 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511662+0000", 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5990000000000001e-06 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511667+0000", 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6110000000000001e-06 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511672+0000", 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5519999999999998e-06 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538456+0000", 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026784175 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538491+0000", 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4683e-05 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538526+0000", 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5457999999999998e-05 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31907 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511678+0000", 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188889571000001, 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026934228000000001, 2024-09-16T22:16:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31907 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511678+0000", 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511678+0000", 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511679+0000", 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511681+0000", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8029999999999999e-06 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511686+0000", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.614e-06 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511691+0000", 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2930000000000002e-06 2024-09-16T22:16:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538547+0000", 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026856367999999999 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538580+0000", 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3308e-05 2024-09-16T22:16:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538613+0000", 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2524000000000002e-05 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31910 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1564672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511696+0000", 2024-09-16T22:16:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188872201000001, 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027007310999999999, 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31910 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511696+0000", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511696+0000", 2024-09-16T22:16:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511696+0000", 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511698+0000", 2024-09-16T22:16:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.778e-06 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511704+0000", 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5520000000000004e-06 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511711+0000", 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4329999999999998e-06 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538634+0000", 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922728999999999 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538667+0000", 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.324e-05 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538703+0000", 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6219e-05 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31913 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1269760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511716+0000", 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188851785000001, 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027075836999999998, 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31913 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511716+0000", 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511716+0000", 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511717+0000", 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511718+0000", 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.66e-06 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511724+0000", 2024-09-16T22:16:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6749999999999999e-06 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511728+0000", 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3699999999999997e-06 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538724+0000", 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026995903000000002 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538757+0000", 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3223999999999999e-05 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538792+0000", 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4653000000000002e-05 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31919 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1990656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511734+0000", 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188834478, 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159868, 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31919 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511734+0000", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511734+0000", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511734+0000", 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511736+0000", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.703e-06 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511741+0000", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7239999999999997e-06 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511746+0000", 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-16T22:16:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538813+0000", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027066844 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538857+0000", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4302999999999999e-05 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538893+0000", 2024-09-16T22:16:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6609000000000001e-05 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31920 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.511751+0000", 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188816983000001, 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027233619000000001, 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31920 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511751+0000", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511751+0000", 2024-09-16T22:16:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511751+0000", 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511753+0000", 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511759+0000", 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049999999999998e-06 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.511763+0000", 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-06 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538916+0000", 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027152813000000001 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538950+0000", 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3633999999999997e-05 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.538985+0000", 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5002999999999997e-05 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31921 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 1646592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512145+0000", 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188423473, 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026934372000000002, 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31921 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512145+0000", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512145+0000", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512145+0000", 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512148+0000", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1360000000000001e-06 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512158+0000", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6330000000000008e-06 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512165+0000", 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2024-09-16T22:16:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539005+0000", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026840605999999999 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539041+0000", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5904e-05 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539079+0000", 2024-09-16T22:16:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7712000000000001e-05 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31924 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512175+0000", 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188392988, 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027001039000000001, 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31924 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512175+0000", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512175+0000", 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512176+0000", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512182+0000", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1870000000000002e-06 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512189+0000", 2024-09-16T22:16:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1189999999999999e-06 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512194+0000", 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2240000000000003e-06 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539103+0000", 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026908751000000002 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539141+0000", 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7647000000000001e-05 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539176+0000", 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.561e-05 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31926 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512246+0000", 2024-09-16T22:16:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188322007, 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027025455, 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31926 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512246+0000", 2024-09-16T22:16:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512246+0000", 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512247+0000", 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512248+0000", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6789999999999999e-06 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512253+0000", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512260+0000", 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0090000000000001e-06 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539200+0000", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026939464999999999 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539236+0000", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6180999999999998e-05 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539272+0000", 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5781000000000002e-05 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31929 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1720320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512267+0000", 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188300678999999, 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027097369999999999, 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31929 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512267+0000", 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512267+0000", 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512268+0000", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0100000000000005e-07 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512270+0000", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0789999999999999e-06 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512276+0000", 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5389999999999998e-06 2024-09-16T22:16:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512281+0000", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3920000000000002e-06 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539294+0000", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027012721999999999 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539330+0000", 2024-09-16T22:16:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5890999999999999e-05 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539365+0000", 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5246000000000001e-05 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31930 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512287+0000", 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188281161999999, 2024-09-16T22:16:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179562000000001, 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31930 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512287+0000", 2024-09-16T22:16:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512287+0000", 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512287+0000", 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512289+0000", 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512295+0000", 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7869999999999998e-06 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512300+0000", 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7829999999999998e-06 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539388+0000", 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027087738 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539424+0000", 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6650999999999999e-05 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539466+0000", 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2259000000000003e-05 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31932 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512306+0000", 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188262073000001, 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027325428999999998, 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31932 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512306+0000", 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512306+0000", 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512306+0000", 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512308+0000", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512314+0000", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3920000000000002e-06 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512319+0000", 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.378e-06 2024-09-16T22:16:54.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539489+0000", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027170060999999999 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539550+0000", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0744e-05 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539631+0000", 2024-09-16T22:16:54.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1689999999999999e-05 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31936 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512325+0000", 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188242823, 2024-09-16T22:16:54.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027511793, 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31936 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512325+0000", 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512325+0000", 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512326+0000", 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512328+0000", 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512334+0000", 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8919999999999997e-06 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512338+0000", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5759999999999999e-06 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539672+0000", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027334089999999998 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539754+0000", 2024-09-16T22:16:54.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1298000000000007e-05 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539837+0000", 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3527000000000002e-05 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31939 3.6 3:64462bd1:::rbd_data.11cd85158340.0000000000000003:d [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512344+0000", 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.188223882999999, 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02768864, 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31939 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512344+0000", 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512344+0000", 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512345+0000", 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7800000000000002e-07 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512346+0000", 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8619999999999999e-06 2024-09-16T22:16:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512352+0000", 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.648e-06 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512356+0000", 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4220000000000002e-06 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539877+0000", 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027520296999999999 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.539956+0000", 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9447e-05 2024-09-16T22:16:54.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.540033+0000", 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6586000000000002e-05 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:31941 3.6 3:6129c3b2:::rbd_data.11cd85158340.0000000000000004:d [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:39.512362+0000", 2024-09-16T22:16:54.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.18820607, 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028318559, 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31941 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512362+0000", 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512362+0000", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512362+0000", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512364+0000", 2024-09-16T22:16:54.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512369+0000", 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3850000000000001e-06 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.512374+0000", 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.540537+0000", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028162840000000001 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.540611+0000", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4617999999999996e-05 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:39.540681+0000", 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9276000000000006e-05 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:32915 3.2 3:421f523a:::rbd_data.11cdcaf424fa.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 3984163~21075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:40.605547+0000", 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.095021003999999, 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033252253000000002, 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32915 2024-09-16T22:16:54.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.605547+0000", 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.605547+0000", 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.605548+0000", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.009e-06 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.613968+0000", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084196989999999992 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.613991+0000", 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3702000000000001e-05 2024-09-16T22:16:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.614018+0000", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6180999999999999e-05 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.614079+0000", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1759000000000003e-05 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.614203+0000", 2024-09-16T22:16:54.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012345999999999999 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.614575+0000", 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000371667 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.632565+0000", 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017990663 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:54.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.638710+0000", 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061443499999999998 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.638750+0000", 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0355000000000002e-05 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:40.638799+0000", 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9407999999999999e-05 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42585 3.7 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5449790490000002, 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052149483000000003, 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42585 2024-09-16T22:16:54.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155591+0000", 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.573e-06 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170506+0000", 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01491556 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170523+0000", 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6545000000000002e-05 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170530+0000", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170624+0000", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3875999999999994e-05 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170759+0000", 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013552799999999999 2024-09-16T22:16:54.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171081+0000", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032176299999999998 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207149+0000", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036067812999999997 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207618+0000", 2024-09-16T22:16:54.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046936500000000002 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207689+0000", 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0827999999999993e-05 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207739+0000", 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9601999999999997e-05 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42587 3.8 3:1432c3a7:::rbd_data.11cd49ca855d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 5767168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5299196730000002, 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037194235999999999, 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42587 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170649+0000", 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.096e-06 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181027+0000", 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010377347 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181047+0000", 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0217999999999999e-05 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181057+0000", 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0142e-05 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181119+0000", 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2163999999999996e-05 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181293+0000", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017324699999999999 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181934+0000", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064154700000000002 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207675+0000", 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025741327000000001 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207750+0000", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4301000000000006e-05 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207804+0000", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4564000000000001e-05 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207843+0000", 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8282999999999999e-05 2024-09-16T22:16:54.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42588 3.2 3:429e3b73:::rbd_data.11cd49ca855d.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.181167+0000", 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5194008950000004, 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030950134000000001, 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:54.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42588 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181167+0000", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181167+0000", 2024-09-16T22:16:54.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181169+0000", 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.533e-06 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.189560+0000", 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083917709999999993 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.189581+0000", 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0953000000000001e-05 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.189593+0000", 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1839e-05 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.189669+0000", 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6103e-05 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.189788+0000", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118331 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.190914+0000", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011260650000000001 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.209899+0000", 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018985565999999999 2024-09-16T22:16:54.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.212057+0000", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021577229999999998 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.212091+0000", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4048999999999999e-05 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.212117+0000", 2024-09-16T22:16:54.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6200999999999999e-05 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:54.802 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:54.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:54.804 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:54.804+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:54.904 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:54.906+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:54.949 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:54.950 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:54.950 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:54.950 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:54.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:55.042 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:55.044+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:55.101 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:55.101 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:55.101 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:55.101 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:55.102 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:55.106 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:55.107+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:55.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:55.207 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:55.209+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:55.256 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:55.256 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18012 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756734941000001, 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026593847, 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18012 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500538+0000", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500539+0000", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1399999999999995e-07 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500542+0000", 2024-09-16T22:16:55.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9110000000000002e-06 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500544+0000", 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3130000000000001e-06 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527056+0000", 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026511349999999999 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527095+0000", 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9530000000000003e-05 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527132+0000", 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6613999999999999e-05 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18013 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:55.261 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756722979999999, 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026682926999999999, 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18013 2024-09-16T22:16:55.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500550+0000", 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3799999999999999e-07 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500551+0000", 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8100000000000001e-07 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500560+0000", 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4010000000000003e-06 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500567+0000", 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4019999999999999e-06 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527155+0000", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026588284 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527195+0000", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.019e-05 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527233+0000", 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7431000000000001e-05 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18014 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756702116, 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026763540999999998, 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18014 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500572+0000", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.085e-06 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500576+0000", 2024-09-16T22:16:55.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8229999999999997e-06 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500578+0000", 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3360000000000002e-06 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527256+0000", 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026677421 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527297+0000", 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1028000000000001e-05 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527334+0000", 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7469000000000003e-05 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18015 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14123008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756691731, 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026853495000000002, 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18015 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500582+0000", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500585+0000", 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9749999999999999e-06 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500587+0000", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527357+0000", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026769826 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527398+0000", 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0329e-05 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527435+0000", 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6968999999999998e-05 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18016 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14204928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756682641000001, 2024-09-16T22:16:55.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026943981999999998, 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18016 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:55.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500591+0000", 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500594+0000", 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500597+0000", 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527457+0000", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026860288 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527497+0000", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9490999999999998e-05 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527534+0000", 2024-09-16T22:16:55.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18017 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756673072000002, 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027035519000000001, 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18017 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500601+0000", 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2200000000000003e-07 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500604+0000", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.959e-06 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500606+0000", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527557+0000", 2024-09-16T22:16:55.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026950941999999999 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527599+0000", 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1962000000000002e-05 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527635+0000", 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18018 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14368768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756664051000001, 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027127043, 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18018 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:55.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500610+0000", 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500613+0000", 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.473e-06 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500616+0000", 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2699999999999999e-06 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527659+0000", 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027043754 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527699+0000", 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9847e-05 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527736+0000", 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6610999999999999e-05 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18019 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14450688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:55.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756653482000001, 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027226185, 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18019 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500620+0000", 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500623+0000", 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.687e-06 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500627+0000", 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3220000000000001e-06 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500630+0000", 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.146e-06 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527758+0000", 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027128635000000002 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527799+0000", 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1009e-05 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527846+0000", 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6173e-05 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18020 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756640161, 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027315971000000001, 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18020 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500634+0000", 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500638+0000", 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8e-06 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500640+0000", 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527869+0000", 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027229410999999999 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527911+0000", 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1402000000000001e-05 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527949+0000", 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7901000000000002e-05 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18021 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756630130000001, 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027409273000000001, 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18021 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.23e-07 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500644+0000", 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3900000000000003e-07 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500647+0000", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500649+0000", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527972+0000", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027322470000000001 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528013+0000", 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1211000000000002e-05 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528052+0000", 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9184000000000003e-05 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18022 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14696448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756618078999999, 2024-09-16T22:16:55.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027498129, 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18022 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:55.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500656+0000", 2024-09-16T22:16:55.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999999e-07 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500659+0000", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2150000000000001e-06 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500665+0000", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528075+0000", 2024-09-16T22:16:55.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027410336 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528116+0000", 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1366999999999999e-05 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528153+0000", 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6640999999999997e-05 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18023 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14778368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756605070999999, 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027586167000000002, 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18023 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:16:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500669+0000", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7900000000000007e-07 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500673+0000", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3200000000000001e-06 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500676+0000", 2024-09-16T22:16:55.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528176+0000", 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027500419000000002 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528217+0000", 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893999999999998e-05 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528254+0000", 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6874e-05 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18024 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14860288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756585780000002, 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665443000000001, 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18024 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2e-07 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500689+0000", 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.207e-06 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500692+0000", 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2780000000000002e-06 2024-09-16T22:16:55.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500694+0000", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528276+0000", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027581847 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528317+0000", 2024-09-16T22:16:55.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0417999999999998e-05 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528352+0000", 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5879e-05 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18025 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756575455, 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02775803, 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18025 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500698+0000", 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500699+0000", 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500701+0000", 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500707+0000", 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528375+0000", 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027668663 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528418+0000", 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2073000000000002e-05 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528455+0000", 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7880000000000003e-05 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18026 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15024128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:55.326 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756562950999999, 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027845890000000002, 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18026 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.327 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500711+0000", 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7499999999999998e-07 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500714+0000", 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500717+0000", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528478+0000", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027761082999999999 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528519+0000", 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1090000000000001e-05 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528556+0000", 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7058999999999999e-05 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18027 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15106048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756553610000001, 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027937167999999998, 2024-09-16T22:16:55.331 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18027 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:55.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500720+0000", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500723+0000", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500725+0000", 2024-09-16T22:16:55.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528578+0000", 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027852946999999999 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528619+0000", 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1069000000000003e-05 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528656+0000", 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6958000000000001e-05 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18028 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15187968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756544305999999, 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028028412999999999, 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18028 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500792+0000", 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3039000000000004e-05 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500793+0000", 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.734e-06 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500798+0000", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3579999999999996e-06 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500800+0000", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8289999999999999e-06 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528680+0000", 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027879095 2024-09-16T22:16:55.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528718+0000", 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8627000000000001e-05 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528757+0000", 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8730999999999999e-05 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18029 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756467303000001, 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028059634999999999, 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18029 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.341 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500975+0000", 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016965599999999999 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500977+0000", 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500983+0000", 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2100000000000001e-06 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500986+0000", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1360000000000001e-06 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528779+0000", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027793629 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528820+0000", 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0689999999999998e-05 2024-09-16T22:16:55.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528865+0000", 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5145000000000002e-05 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18030 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756281232999999, 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027977591, 2024-09-16T22:16:55.345 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18030 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501041+0000", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9799000000000002e-05 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501047+0000", 2024-09-16T22:16:55.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0009999999999998e-06 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501052+0000", 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501055+0000", 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528890+0000", 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027835642000000001 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528930+0000", 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0076000000000001e-05 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528969+0000", 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8741000000000001e-05 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18031 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15433728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.756212910999999, 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028011306999999999, 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18031 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505887+0000", 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048273159999999999 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505891+0000", 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.36e-06 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505897+0000", 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7440000000000001e-06 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505908+0000", 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0421000000000001e-05 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528993+0000", 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023084957 2024-09-16T22:16:55.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529034+0000", 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1023999999999999e-05 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529071+0000", 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7484999999999998e-05 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.354 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:55.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:16:55.356 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone23 from 0x986e800 (overlap 0xa386be) to 0x105a46 2024-09-16T22:16:55.357 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:55.356+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:55.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:55.402 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T22:16:55.415 INFO:teuthology.orchestra.run.smithi031.stdout:2703 trunc from 0x986e800 to 0x27e7a3d 2024-09-16T22:16:55.456 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:55.458+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:55.471 INFO:teuthology.orchestra.run.smithi031.stdout:2704 punch from 0x1f890a2 to 0x1f8c8dd, (0x383b bytes) 2024-09-16T22:16:55.487 INFO:teuthology.orchestra.run.smithi031.stdout:2706 write 0xb863341 thru 0xb869697 (0x6357 bytes) 2024-09-16T22:16:55.497 INFO:teuthology.orchestra.run.smithi031.stdout:2707 trunc from 0xb869698 to 0xeb4647d 2024-09-16T22:16:55.502 INFO:teuthology.orchestra.run.smithi031.stdout:2710 writesame 0x755caa thru 0x75fa1e data_size 0x1(0x9d75 bytes) 2024-09-16T22:16:55.506 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:55.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:55.506 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:55.506 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:55.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:55.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:16:55.557 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:55.559+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:55.580 INFO:teuthology.orchestra.run.smithi031.stdout:2711 write 0xb55e5c4 thru 0xb56d9e1 (0xf41e bytes) 2024-09-16T22:16:55.600 INFO:teuthology.orchestra.run.smithi031.stdout:2713 trunc from 0xeb4647d to 0x42d4ff9 2024-09-16T22:16:55.612 INFO:teuthology.orchestra.run.smithi031.stdout:2714 write 0x16c6871 thru 0x16d4e44 (0xe5d4 bytes) 2024-09-16T22:16:55.657 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:55.658+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:55.660 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:55.661 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:55.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:55.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:55.684 INFO:teuthology.orchestra.run.smithi031.stdout:2715 write 0xbe47334 thru 0xbe4f256 (0x7f23 bytes) 2024-09-16T22:16:55.695 INFO:teuthology.orchestra.run.smithi031.stdout:2716 read 0x1ff9e7e thru 0x200939f (0xf522 bytes) 2024-09-16T22:16:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:2718 writesame 0x437897b thru 0x43873d1 data_size 0x1(0xea57 bytes) 2024-09-16T22:16:55.706 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:55.706 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:55.723 INFO:teuthology.orchestra.run.smithi031.stdout:2719 trunc from 0xbe4f257 to 0x5f6d405 2024-09-16T22:16:55.726 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:55.726 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:55.734 INFO:teuthology.orchestra.run.smithi031.stdout:2720 punch from 0x1d6911 to 0x1d93a9, (0x2a98 bytes) 2024-09-16T22:16:55.735 INFO:teuthology.orchestra.run.smithi031.stdout:2721 write 0x565e658 thru 0x566a725 (0xc0ce bytes) 2024-09-16T22:16:55.745 INFO:teuthology.orchestra.run.smithi031.stdout:2722 trunc from 0x5f6d405 to 0x2dc1ec8 2024-09-16T22:16:55.750 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:55.750 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:55.758 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:55.759+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:55.763 INFO:teuthology.orchestra.run.smithi031.stdout:2723 write 0x2f06822 thru 0x2f0e0e8 (0x78c7 bytes) 2024-09-16T22:16:55.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:55.772 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:55.772 INFO:teuthology.orchestra.run.smithi031.stdout:2724 punch from 0x3d981b to 0x3dc953, (0x3138 bytes) 2024-09-16T22:16:55.772 INFO:teuthology.orchestra.run.smithi031.stdout:2725 clone 25 order 22 su 2097152 sc 5 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.186166471, 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:16:55.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:16:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:55.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.595258922999999, 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:16:55.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:16:55.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:16:55.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.544744233999999, 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:16:55.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.531418144, 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:16:55.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:16:55.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.189919631999999, 2024-09-16T22:16:55.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:16:55.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:16:55.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:33503 3.1 3:84128263:::rbd_data.11cd14063e78.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.131651678000001, 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021447790000000001, 2024-09-16T22:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33503 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647296+0000", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0500000000000002e-07 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647306+0000", 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0061000000000001e-05 2024-09-16T22:16:55.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647317+0000", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0502e-05 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647338+0000", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880999999999999e-05 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647393+0000", 2024-09-16T22:16:55.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4376999999999998e-05 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647469+0000", 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6044000000000006e-05 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.650937+0000", 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003467932 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:55.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668661+0000", 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017723869 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668674+0000", 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3251000000000001e-05 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668707+0000", 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3599000000000001e-05 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668743+0000", 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5469000000000002e-05 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.476569618999999, 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:16:55.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42341 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.835094915, 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097858919999999992, 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42341 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:55.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.11e-07 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943853+0000", 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2099999999999995e-07 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943856+0000", 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8569999999999999e-06 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943858+0000", 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3410000000000001e-06 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953521+0000", 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009662314 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953579+0000", 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8876999999999999e-05 2024-09-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953638+0000", 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8371000000000001e-05 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42349 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.8350857659999997, 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099116080000000006, 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42349 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:55.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0699999999999999e-07 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943862+0000", 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.33e-07 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943865+0000", 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.914e-06 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943867+0000", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953669+0000", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098011109999999995 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7537999999999997e-05 2024-09-16T22:16:55.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953773+0000", 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6567000000000003e-05 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42353 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.8350764850000001, 2024-09-16T22:16:55.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010048461, 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42353 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943871+0000", 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943872+0000", 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3099999999999996e-07 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943875+0000", 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4450000000000001e-06 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943883+0000", 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3559999999999993e-06 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953791+0000", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099081020000000002 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953862+0000", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1050999999999994e-05 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953919+0000", 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6366999999999998e-05 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42585 3.7 e51/23 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head v 51'156, mlcod=51'154)", 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.607553598, 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036096402999999999, 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171395+0000", 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174324+0000", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029293370000000002 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174349+0000", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174358+0000", 2024-09-16T22:16:55.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6979999999999993e-06 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174442+0000", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3858999999999998e-05 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174470+0000", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8042999999999999e-05 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207434+0000", 2024-09-16T22:16:55.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032963615000000002 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207490+0000", 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831999999999997e-05 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44663 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:55.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2170759369999997, 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011080998999999999, 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44663 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0499999999999999e-07 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561879+0000", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5349999999999999e-06 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561886+0000", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2590000000000003e-06 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561892+0000", 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6389999999999997e-06 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570230+0000", 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.008338715 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570292+0000", 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1756999999999998e-05 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572952+0000", 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026596889999999998 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44776 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2168554660000002, 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098382499999999998, 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44776 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562092+0000", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562094+0000", 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562101+0000", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.024e-06 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562115+0000", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4054e-05 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571783+0000", 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096675010000000002 2024-09-16T22:16:55.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571850+0000", 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.737e-05 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571930+0000", 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9476999999999998e-05 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44777 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 77824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2168188300000002, 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099730849999999996, 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44777 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562129+0000", 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.51e-07 2024-09-16T22:16:55.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562131+0000", 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562138+0000", 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562144+0000", 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049999999999998e-06 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571953+0000", 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098094210000000005 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572022+0000", 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9190000000000007e-05 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572101+0000", 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8956999999999996e-05 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44778 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2167906769999997, 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010114365, 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44778 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:55.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562157+0000", 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562158+0000", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562165+0000", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562170+0000", 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572125+0000", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099544479999999994 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572193+0000", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8262000000000005e-05 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572271+0000", 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7411999999999997e-05 2024-09-16T22:16:55.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44779 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2167683870000001, 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010260511999999999, 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44779 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562179+0000", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562185+0000", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6740000000000002e-06 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562191+0000", 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9410000000000001e-06 2024-09-16T22:16:55.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562197+0000", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572297+0000", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010099542 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572361+0000", 2024-09-16T22:16:55.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4673999999999998e-05 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572439+0000", 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7717000000000001e-05 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44780 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2167426710000004, 2024-09-16T22:16:55.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01039753, 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44780 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:55.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562205+0000", 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562207+0000", 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.956e-06 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562213+0000", 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5799999999999997e-06 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562218+0000", 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-06 2024-09-16T22:16:55.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572461+0000", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010243321999999999 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572526+0000", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4292e-05 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572602+0000", 2024-09-16T22:16:55.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5952999999999996e-05 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44781 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2167217670000001, 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010560442, 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44781 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:55.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562228+0000", 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562234+0000", 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562239+0000", 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0080000000000002e-06 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572626+0000", 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010386799 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572693+0000", 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7008999999999995e-05 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572786+0000", 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2346e-05 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.876 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44782 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2166840409999997, 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010842269, 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44782 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9299999999999998e-07 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562266+0000", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562273+0000", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0670000000000003e-06 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562279+0000", 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:16:55.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572810+0000", 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010530684 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572944+0000", 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134387 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573105+0000", 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000161096 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44783 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2166600279999997, 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011016781999999999, 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:55.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44783 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:55.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562290+0000", 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2579999999999998e-06 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562296+0000", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562301+0000", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573160+0000", 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010858945 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573221+0000", 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0402000000000002e-05 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573304+0000", 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2807999999999997e-05 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:55.886 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:55.886 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:55.887 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:55.887 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:55.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:55.889 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:55.889 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:55.890 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:55.887+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:55.988 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:55.989+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:55.991 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:55.991 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:55.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:16:56.089 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:56.090+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:56.101 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.102 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:16:56.127 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:16:56.195 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:56.197+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:56.253 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.253 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:16:56.305 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:56.306+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:56.373 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.373 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:16:56.391 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:56.393+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:56.492 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:56.494+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:56.528 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.528 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:16:56.593 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:56.593+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:56.644 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.644 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:16:56.694 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:56.696+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:56.795 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:56.797+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:56.801 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:56.801 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:56.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:16:56.897 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:56.898+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:57.234 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:57.235 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:57.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:16:57.235 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone24 from 0x2f0e0e9 (overlap 0x27e7a3d) to 0x13f3d1e 2024-09-16T22:16:57.236 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:57.236+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:57.335 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:57.336+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:57.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:16:57.453 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:57.454+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:57.532 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:57.533 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:57.536 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:57.537+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:57.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:16:57.636 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:57.638+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:57.701 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:57.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:16:57.737 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:16:57.739+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:16:57.741 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T22:16:57.747 INFO:teuthology.orchestra.run.smithi031.stdout:2726 read 0x17a8710 thru 0x17a9d82 (0x1673 bytes) 2024-09-16T22:16:57.748 INFO:teuthology.orchestra.run.smithi031.stdout:2727 read 0x19ad2b1 thru 0x19bd116 (0xfe66 bytes) 2024-09-16T22:16:57.749 INFO:teuthology.orchestra.run.smithi031.stdout:2728 read 0x862ec9 thru 0x867d4c (0x4e84 bytes) 2024-09-16T22:16:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:57.750 INFO:teuthology.orchestra.run.smithi031.stdout:2731 punch from 0xf3abac to 0xf3c7e4, (0x1c38 bytes) 2024-09-16T22:16:57.753 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:57.754 INFO:teuthology.orchestra.run.smithi031.stdout:2734 read 0x92bbf4 thru 0x93988a (0xdc97 bytes) 2024-09-16T22:16:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:2738 read 0x1795f34 thru 0x179a286 (0x4353 bytes) 2024-09-16T22:16:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:2739 read 0x19ac65f thru 0x19b3e87 (0x7829 bytes) 2024-09-16T22:16:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:2741 read 0x11bf9f0 thru 0x11c29da (0x2feb bytes) 2024-09-16T22:16:57.757 INFO:teuthology.orchestra.run.smithi031.stdout:2742 punch from 0x1bc8240 to 0x1bd0a35, (0x87f5 bytes) 2024-09-16T22:16:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:2743 write 0x67ab580 thru 0x67ae6e1 (0x3162 bytes) 2024-09-16T22:16:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:2745 write 0xe0eb5e5 thru 0xe0ebdd0 (0x7ec bytes) 2024-09-16T22:16:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:2746 read 0xebf1f1 thru 0xecb755 (0xc565 bytes) 2024-09-16T22:16:57.791 INFO:teuthology.orchestra.run.smithi031.stdout:2747 write 0xcb98577 thru 0xcb993ab (0xe35 bytes) 2024-09-16T22:16:57.794 INFO:teuthology.orchestra.run.smithi031.stdout:2749 write 0x19a1a9f thru 0x19a732c (0x588e bytes) 2024-09-16T22:16:57.836 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:57.837 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:57.839 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:16:57.839+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:16:57.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:16:57.889 INFO:teuthology.orchestra.run.smithi031.stdout:2751 writesame 0x2d618d9 thru 0x2d65ad9 data_size 0x1(0x4201 bytes) 2024-09-16T22:16:57.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:16:57.898 INFO:teuthology.orchestra.run.smithi031.stdout:2752 writesame 0x652df0f thru 0x653b0f7 data_size 0x1d(0xd1e9 bytes) 2024-09-16T22:16:57.903 INFO:teuthology.orchestra.run.smithi031.stdout:2753 writesame 0xcc1a822 thru 0xcc20cdf data_size 0x5(0x64be bytes) 2024-09-16T22:16:57.908 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:57.914 INFO:teuthology.orchestra.run.smithi031.stdout:2757 writesame 0xe37958c thru 0xe385a1a data_size 0x3(0xc48f bytes) 2024-09-16T22:16:57.914 INFO:teuthology.orchestra.run.smithi031.stdout:2758 write 0x824d193 thru 0x8257906 (0xa774 bytes) 2024-09-16T22:16:57.918 INFO:teuthology.orchestra.run.smithi031.stdout:2759 read 0x66a1d8e thru 0x66a1f98 (0x20b bytes) 2024-09-16T22:16:57.922 INFO:teuthology.orchestra.run.smithi031.stdout:2760 trunc from 0xe385a1b to 0xad42831 2024-09-16T22:16:57.933 INFO:teuthology.orchestra.run.smithi031.stdout:2761 write 0x9f6d05 thru 0x9f7d16 (0x1012 bytes) 2024-09-16T22:16:57.940 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:57.941+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:58.035 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:16:58.036 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:16:58.041 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:58.041+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:58.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:16:58.064 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:16:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:16:58.080 INFO:teuthology.orchestra.run.smithi031.stdout:2762 trunc from 0xad42831 to 0xc52af0c 2024-09-16T22:16:58.083 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:16:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:16:58.087 INFO:teuthology.orchestra.run.smithi031.stdout:2763 read 0x567a783 thru 0x5681ab8 (0x7336 bytes) 2024-09-16T22:16:58.095 INFO:teuthology.orchestra.run.smithi031.stdout:2764 write 0xe4dfec2 thru 0xe4ebd2e (0xbe6d bytes) 2024-09-16T22:16:58.098 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:58.098 INFO:teuthology.orchestra.run.smithi031.stdout:2767 writesame 0x2aea477 thru 0x2aee034 data_size 0x3(0x3bbe bytes) 2024-09-16T22:16:58.103 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:16:58.104 INFO:teuthology.orchestra.run.smithi031.stdout:2768 write 0x4a23c7e thru 0x4a31af0 (0xde73 bytes) 2024-09-16T22:16:58.107 INFO:teuthology.orchestra.run.smithi031.stdout:2769 write 0x7b59349 thru 0x7b5fbaf (0x6867 bytes) 2024-09-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:58.110 INFO:teuthology.orchestra.run.smithi031.stdout:2772 writesame 0x82ad31a thru 0x82b974d data_size 0x3a(0xc434 bytes) 2024-09-16T22:16:58.112 INFO:teuthology.orchestra.run.smithi031.stdout:2773 read 0x5f84e74 thru 0x5f86dde (0x1f6b bytes) 2024-09-16T22:16:58.112 INFO:teuthology.orchestra.run.smithi031.stdout:2774 clone 26 order 22 su 2097152 sc 5 2024-09-16T22:16:58.123 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:16:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:16:58.141 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:58.142+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:58.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:16:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:16:58.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:16:58.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:16:58.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:16:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:16:58.192 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:16:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:16:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:16:58.193 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:58.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:16:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:16:58.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:16:58.242 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:58.244+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:58.249 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone25 2024-09-16T22:16:58.335 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:16:58.335 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:16:58.335 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-16T22:16:58.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-16T22:16:58.345 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:58.347+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:58.422 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.995516874000003, 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:16:58.423 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:58.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:16:58.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.160711822, 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.433 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.16069098300000001, 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:16:58.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.160667743, 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:16:58.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:16:58.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15996585799999999, 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:16:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15683341100000001, 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:16:58.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.156423801, 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:16:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.156137624, 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:16:58.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15584039999999999, 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:16:58.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.155052157, 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:16:58.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15493738800000001, 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:16:58.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:16:58.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15468617100000001, 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:16:58.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:16:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:16:58.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.154377813, 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:16:58.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.484 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15433887600000001, 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:16:58.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15405345600000001, 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:16:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.148201953, 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:16:58.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:16:58.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14817755499999999, 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:16:58.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.148126905, 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:16:58.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.148093208, 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:16:58.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14798223799999999, 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:16:58.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:16:58.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:16:58.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:16:58.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:16:58.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:16:58.517 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:16:58.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:16:58.519 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:58.519+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:58.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-16T22:16:58.622 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:58.623+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:58.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:58.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:58.623 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:58.623 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:58.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:16:58.720 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:58.722+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:58.734 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:58.734 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:58.734 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:58.735 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:58.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:16:58.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-16T22:16:58.833 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:16:58.834+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42585 3.7 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:58.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7520142669999998, 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052149483000000003, 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42585 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155591+0000", 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.573e-06 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170506+0000", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01491556 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170523+0000", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6545000000000002e-05 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170530+0000", 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:16:58.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170624+0000", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3875999999999994e-05 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170759+0000", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013552799999999999 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171081+0000", 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032176299999999998 2024-09-16T22:16:58.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207149+0000", 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036067812999999997 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207618+0000", 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046936500000000002 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:58.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207689+0000", 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0827999999999993e-05 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207739+0000", 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9601999999999997e-05 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42587 3.8 3:1432c3a7:::rbd_data.11cd49ca855d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 5767168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:58.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7369548909999999, 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037194235999999999, 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42587 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170649+0000", 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.096e-06 2024-09-16T22:16:58.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181027+0000", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010377347 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181047+0000", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0217999999999999e-05 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181057+0000", 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0142e-05 2024-09-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181119+0000", 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2163999999999996e-05 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181293+0000", 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017324699999999999 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181934+0000", 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064154700000000002 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207675+0000", 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025741327000000001 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207750+0000", 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4301000000000006e-05 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207804+0000", 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4564000000000001e-05 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207843+0000", 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8282999999999999e-05 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59223 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6328320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.052204278999999999, 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034753091999999999, 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59223 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855484+0000", 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5171000000000002e-05 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855487+0000", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855494+0000", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5059999999999998e-06 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855499+0000", 2024-09-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9250000000000003e-06 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890044+0000", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034544856999999998 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890094+0000", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9632999999999997e-05 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890152+0000", 2024-09-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8329999999999999e-05 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59224 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6373376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.052094698000000002, 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034745103999999999, 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59224 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855695+0000", 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000186205 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855697+0000", 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.481e-06 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855702+0000", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855706+0000", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8510000000000001e-06 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890164+0000", 2024-09-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034458216 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890204+0000", 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9756999999999999e-05 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890254+0000", 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9916000000000001e-05 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59229 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6377472~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.051431987999999998, 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034688831000000003, 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59229 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856260+0000", 2024-09-16T22:16:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8486000000000006e-05 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856266+0000", 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6000000000000003e-06 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856273+0000", 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5629999999999997e-06 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856280+0000", 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8090000000000004e-06 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890761+0000", 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034480982 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890802+0000", 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1625000000000002e-05 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890860+0000", 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7766000000000002e-05 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59262 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6684672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.047067573000000001, 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034901661, 2024-09-16T22:16:58.933 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59262 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860744+0000", 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208519 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860747+0000", 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5210000000000001e-06 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860752+0000", 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860758+0000", 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8479999999999997e-06 2024-09-16T22:16:58.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895226+0000", 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034467822000000002 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895320+0000", 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4729999999999996e-05 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895437+0000", 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011711599999999999 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59263 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6656000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.046821261000000003, 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034894085999999998, 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59263 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860783+0000", 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860785+0000", 2024-09-16T22:16:58.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839999999999999e-06 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860791+0000", 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8390000000000003e-06 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860796+0000", 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8139999999999997e-06 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895487+0000", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034690152000000002 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895579+0000", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2514000000000002e-05 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895676+0000", 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7060999999999998e-05 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59264 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6701056~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.046798329, 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034991059999999997, 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59264 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:16:58.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862891+0000", 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002086409 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862895+0000", 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0110000000000002e-06 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862904+0000", 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0920000000000004e-06 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862911+0000", 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4049999999999997e-06 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895691+0000", 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032779642999999997 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895744+0000", 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3767000000000002e-05 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895796+0000", 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1733000000000001e-05 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59275 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6754304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044455878999999997, 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034872215999999998, 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59275 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863148+0000", 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:16:58.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863149+0000", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1000000000000001e-06 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863152+0000", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0369999999999999e-06 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863154+0000", 2024-09-16T22:16:58.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.897823+0000", 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034668869999999997 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.897915+0000", 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1735000000000002e-05 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898020+0000", 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104777 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59276 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6799360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044446052, 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034967300999999999, 2024-09-16T22:16:58.952 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59276 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863159+0000", 2024-09-16T22:16:58.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1629999999999999e-06 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863162+0000", 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249999999999999e-06 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863164+0000", 2024-09-16T22:16:58.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5849999999999998e-06 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898033+0000", 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034868561999999999 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898076+0000", 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3163e-05 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898125+0000", 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8683000000000001e-05 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59277 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6770688~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044435529000000001, 2024-09-16T22:16:58.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035191287000000002, 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59277 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863175+0000", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0360000000000001e-06 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863178+0000", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4340000000000001e-06 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863181+0000", 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.695e-06 2024-09-16T22:16:58.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898161+0000", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034980306000000003 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898256+0000", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4986000000000002e-05 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898359+0000", 2024-09-16T22:16:58.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102606 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59278 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6815744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044418766999999998, 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035275754999999999, 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59278 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863186+0000", 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.203e-06 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863189+0000", 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.948e-06 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863191+0000", 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5450000000000002e-06 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898372+0000", 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035180077999999997 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898411+0000", 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9067999999999999e-05 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898460+0000", 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9691999999999998e-05 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59279 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6787072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044408956999999999, 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03549563, 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59279 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863195+0000", 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:16:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863196+0000", 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249999999999999e-06 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863199+0000", 2024-09-16T22:16:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2059999999999999e-06 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863204+0000", 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7779999999999998e-06 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898497+0000", 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035293195999999999 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898592+0000", 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5228000000000004e-05 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898690+0000", 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7948999999999993e-05 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59280 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6832128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044383578999999999, 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035629223000000002, 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59280 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863222+0000", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0899999999999999e-06 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863228+0000", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863235+0000", 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7959999999999998e-06 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898723+0000", 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035488820999999997 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898775+0000", 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1425999999999999e-05 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898849+0000", 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4175e-05 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59281 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6803456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044361151000000001, 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035848440000000002, 2024-09-16T22:16:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59281 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863243+0000", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863245+0000", 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.074e-06 2024-09-16T22:16:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863251+0000", 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8539999999999999e-06 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863258+0000", 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1299999999999998e-06 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898889+0000", 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035631384000000002 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898987+0000", 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8062999999999999e-05 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899091+0000", 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103498 2024-09-16T22:16:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59282 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6848512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044339127999999998, 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035937208999999998, 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59282 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863265+0000", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863266+0000", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863275+0000", 2024-09-16T22:16:58.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863281+0000", 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0100000000000001e-06 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899106+0000", 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035825203999999999 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899148+0000", 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2233e-05 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899201+0000", 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3340000000000001e-05 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59283 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6819840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:16:58.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044309448000000001, 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036143469999999997, 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59283 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:16:58.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863296+0000", 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1469999999999999e-06 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863304+0000", 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7049999999999999e-06 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863310+0000", 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6949999999999997e-06 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899239+0000", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035928845000000001 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899335+0000", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5872000000000006e-05 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899437+0000", 2024-09-16T22:16:58.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102746 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59284 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6864896~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044285181999999999, 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036222813999999999, 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59284 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863519+0000", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020047899999999999 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863522+0000", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863529+0000", 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9090000000000003e-06 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863535+0000", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6080000000000001e-06 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899451+0000", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035915646000000002 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899491+0000", 2024-09-16T22:16:58.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0188000000000002e-05 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899541+0000", 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9824000000000003e-05 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59285 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6836224~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044058195000000001, 2024-09-16T22:16:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036230115, 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59285 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:16:58.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863546+0000", 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:58.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863547+0000", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863556+0000", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6819999999999992e-06 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863561+0000", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899579+0000", 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036018224000000001 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899672+0000", 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2588000000000002e-05 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899775+0000", 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103447 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:58.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59286 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6881280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.044033426000000001, 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036317443999999997, 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59286 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868940+0000", 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053705530000000001 2024-09-16T22:16:59.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868947+0000", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3589999999999996e-06 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868956+0000", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1700000000000003e-06 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868964+0000", 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-09-16T22:16:59.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899789+0000", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030824579000000001 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899834+0000", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5478000000000001e-05 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899887+0000", 2024-09-16T22:16:59.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3155999999999997e-05 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.004 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:16:59.006 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:59.006+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:59.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-16T22:16:59.106 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:59.107+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:59.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:59.109 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:59.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:16:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:16:59.198 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T22:16:59.207 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:16:59.208+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:16:59.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:59.224 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:16:59.308 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:59.310+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18014 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.337 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837571711999999, 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026763540999999998, 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18014 2024-09-16T22:16:59.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500572+0000", 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.085e-06 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500576+0000", 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8229999999999997e-06 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500578+0000", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3360000000000002e-06 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527256+0000", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026677421 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527297+0000", 2024-09-16T22:16:59.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1028000000000001e-05 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527334+0000", 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7469000000000003e-05 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18015 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14123008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.342 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837561327, 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026853495000000002, 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18015 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-09-16T22:16:59.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500582+0000", 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500585+0000", 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9749999999999999e-06 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500587+0000", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527357+0000", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026769826 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527398+0000", 2024-09-16T22:16:59.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0329e-05 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527435+0000", 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6968999999999998e-05 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18016 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14204928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:59.347 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837552237000001, 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026943981999999998, 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18016 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500591+0000", 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500594+0000", 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500597+0000", 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527457+0000", 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026860288 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527497+0000", 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9490999999999998e-05 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527534+0000", 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18017 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837542668000001, 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027035519000000001, 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18017 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:16:59.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500601+0000", 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2200000000000003e-07 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500604+0000", 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.959e-06 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500606+0000", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527557+0000", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026950941999999999 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527599+0000", 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1962000000000002e-05 2024-09-16T22:16:59.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527635+0000", 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18018 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14368768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:59.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837533647000001, 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027127043, 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18018 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500610+0000", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500613+0000", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.473e-06 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500616+0000", 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2699999999999999e-06 2024-09-16T22:16:59.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527659+0000", 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027043754 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527699+0000", 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9847e-05 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527736+0000", 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6610999999999999e-05 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18019 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14450688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837523078, 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027226185, 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18019 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500620+0000", 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500623+0000", 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.687e-06 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500627+0000", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3220000000000001e-06 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500630+0000", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.146e-06 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527758+0000", 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027128635000000002 2024-09-16T22:16:59.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527799+0000", 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1009e-05 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527846+0000", 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6173e-05 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18020 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837509756999999, 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027315971000000001, 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18020 2024-09-16T22:16:59.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500634+0000", 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500638+0000", 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8e-06 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500640+0000", 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527869+0000", 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027229410999999999 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527911+0000", 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1402000000000001e-05 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527949+0000", 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7901000000000002e-05 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18021 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837499726000001, 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027409273000000001, 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18021 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.23e-07 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500644+0000", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3900000000000003e-07 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500647+0000", 2024-09-16T22:16:59.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500649+0000", 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527972+0000", 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027322470000000001 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528013+0000", 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1211000000000002e-05 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528052+0000", 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9184000000000003e-05 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18022 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14696448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837487674999998, 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027498129, 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18022 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.377 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500656+0000", 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999999e-07 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500659+0000", 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2150000000000001e-06 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500665+0000", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528075+0000", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027410336 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528116+0000", 2024-09-16T22:16:59.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1366999999999999e-05 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528153+0000", 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6640999999999997e-05 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18023 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14778368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837474666999999, 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027586167000000002, 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18023 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500669+0000", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7900000000000007e-07 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500673+0000", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3200000000000001e-06 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500676+0000", 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:16:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528176+0000", 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027500419000000002 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528217+0000", 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893999999999998e-05 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528254+0000", 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6874e-05 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18024 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14860288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837455376000001, 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665443000000001, 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18024 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2e-07 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500689+0000", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.207e-06 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500692+0000", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2780000000000002e-06 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500694+0000", 2024-09-16T22:16:59.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528276+0000", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027581847 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528317+0000", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0417999999999998e-05 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528352+0000", 2024-09-16T22:16:59.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5879e-05 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18025 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837445051, 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02775803, 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18025 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500698+0000", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500699+0000", 2024-09-16T22:16:59.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500701+0000", 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500707+0000", 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528375+0000", 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027668663 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528418+0000", 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2073000000000002e-05 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528455+0000", 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7880000000000003e-05 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18026 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15024128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837432546999999, 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027845890000000002, 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18026 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500711+0000", 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7499999999999998e-07 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500714+0000", 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500717+0000", 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528478+0000", 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027761082999999999 2024-09-16T22:16:59.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528519+0000", 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1090000000000001e-05 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528556+0000", 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7058999999999999e-05 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18027 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15106048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837423206, 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027937167999999998, 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18027 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500720+0000", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500723+0000", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500725+0000", 2024-09-16T22:16:59.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528578+0000", 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027852946999999999 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528619+0000", 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1069000000000003e-05 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528656+0000", 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6958000000000001e-05 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18028 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15187968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837413902000002, 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028028412999999999, 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18028 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500792+0000", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3039000000000004e-05 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500793+0000", 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.734e-06 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500798+0000", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3579999999999996e-06 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500800+0000", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8289999999999999e-06 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528680+0000", 2024-09-16T22:16:59.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027879095 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528718+0000", 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8627000000000001e-05 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528757+0000", 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8730999999999999e-05 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18029 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837336899, 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028059634999999999, 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18029 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500975+0000", 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016965599999999999 2024-09-16T22:16:59.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500977+0000", 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500983+0000", 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2100000000000001e-06 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500986+0000", 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1360000000000001e-06 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528779+0000", 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027793629 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528820+0000", 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0689999999999998e-05 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528865+0000", 2024-09-16T22:16:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5145000000000002e-05 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18030 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837150828999999, 2024-09-16T22:16:59.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027977591, 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18030 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501041+0000", 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9799000000000002e-05 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501047+0000", 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0009999999999998e-06 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501052+0000", 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501055+0000", 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528890+0000", 2024-09-16T22:16:59.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027835642000000001 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528930+0000", 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0076000000000001e-05 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528969+0000", 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8741000000000001e-05 2024-09-16T22:16:59.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18031 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15433728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.837082507000002, 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028011306999999999, 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.423 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18031 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505887+0000", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048273159999999999 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505891+0000", 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.36e-06 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505897+0000", 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7440000000000001e-06 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505908+0000", 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0421000000000001e-05 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528993+0000", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023084957 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529034+0000", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1023999999999999e-05 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529071+0000", 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7484999999999998e-05 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59481 3.0 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.40608001300000002, 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034053075000000002, 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.428 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59481 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:16:59.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932064+0000", 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.883e-06 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.936978+0000", 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049139240000000001 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.936996+0000", 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8209000000000001e-05 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937004+0000", 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4220000000000003e-06 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937061+0000", 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7138000000000001e-05 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937157+0000", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6086000000000001e-05 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937467+0000", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000310396 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.949047+0000", 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011579325 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966042+0000", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016995415999999999 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966082+0000", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0046999999999998e-05 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966115+0000", 2024-09-16T22:16:59.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3229000000000003e-05 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59482 3.0 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head [write 0~0] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.40111729000000002, 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030075369000000001, 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59482 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937026+0000", 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6400000000000002e-07 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937028+0000", 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.979e-06 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937032+0000", 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937036+0000", 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8589999999999998e-06 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943162+0000", 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061266439999999997 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943248+0000", 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5612999999999995e-05 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943390+0000", 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142552 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950442+0000", 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070514250000000001 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967043+0000", 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016601405999999999 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967075+0000", 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1585000000000003e-05 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967100+0000", 2024-09-16T22:16:59.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582e-05 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.442 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:16:59.445 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:16:59.445+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:16:59.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-16T22:16:59.545 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:59.546+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:59.588 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.588 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:59.588 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:16:59.588 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:16:59.646 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:16:59.647+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:16:59.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-16T22:16:59.701 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.701 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:16:59.701 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:16:59.701 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:16:59.701 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:16:59.754 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:16:59.756+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:16:59.794 INFO:teuthology.orchestra.run.smithi031.stdout:2775 read 0x1e04df9 thru 0x1e062d1 (0x14d9 bytes) 2024-09-16T22:16:59.795 INFO:teuthology.orchestra.run.smithi031.stdout:2776 write 0x7574aea thru 0x757fef1 (0xb408 bytes) 2024-09-16T22:16:59.811 INFO:teuthology.orchestra.run.smithi031.stdout:2777 trunc from 0xe4ebd2f to 0x6843f9e 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:16:59.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.223902979999998, 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:16:59.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:59.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.632995432000001, 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:16:59.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.582480742999998, 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:59.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.569154652999998, 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:16:59.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.227656141000001, 2024-09-16T22:16:59.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:16:59.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:33503 3.1 3:84128263:::rbd_data.11cd14063e78.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.169388186999999, 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021447790000000001, 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33503 2024-09-16T22:16:59.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647296+0000", 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0500000000000002e-07 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647306+0000", 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0061000000000001e-05 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647317+0000", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0502e-05 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647338+0000", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880999999999999e-05 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647393+0000", 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4376999999999998e-05 2024-09-16T22:16:59.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647469+0000", 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6044000000000006e-05 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.650937+0000", 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003467932 2024-09-16T22:16:59.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668661+0000", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017723869 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668674+0000", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3251000000000001e-05 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668707+0000", 2024-09-16T22:16:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3599000000000001e-05 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668743+0000", 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5469000000000002e-05 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.514306127999999, 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.856 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:16:59.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:16:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:16:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:16:59.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:16:59.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42341 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.872831423999999, 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097858919999999992, 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42341 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943852+0000", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.11e-07 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943853+0000", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2099999999999995e-07 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943856+0000", 2024-09-16T22:16:59.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8569999999999999e-06 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943858+0000", 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3410000000000001e-06 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953521+0000", 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009662314 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953579+0000", 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8876999999999999e-05 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953638+0000", 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8371000000000001e-05 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42349 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:59.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.872822275000001, 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099116080000000006, 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42349 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0699999999999999e-07 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943862+0000", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.33e-07 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943865+0000", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.914e-06 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943867+0000", 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-09-16T22:16:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953669+0000", 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098011109999999995 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7537999999999997e-05 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953773+0000", 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6567000000000003e-05 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42353 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.872812994, 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010048461, 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42353 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943871+0000", 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943872+0000", 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3099999999999996e-07 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943875+0000", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4450000000000001e-06 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943883+0000", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3559999999999993e-06 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953791+0000", 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099081020000000002 2024-09-16T22:16:59.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953862+0000", 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1050999999999994e-05 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953919+0000", 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6366999999999998e-05 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42585 3.7 e51/23 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head v 51'156, mlcod=51'154)", 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.645290106999999, 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036096402999999999, 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171395+0000", 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174324+0000", 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029293370000000002 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174349+0000", 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174358+0000", 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6979999999999993e-06 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174442+0000", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3858999999999998e-05 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174470+0000", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8042999999999999e-05 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207434+0000", 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032963615000000002 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207490+0000", 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831999999999997e-05 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44663 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2548124460000007, 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011080998999999999, 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44663 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0499999999999999e-07 2024-09-16T22:16:59.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561879+0000", 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5349999999999999e-06 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561886+0000", 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2590000000000003e-06 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561892+0000", 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6389999999999997e-06 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570230+0000", 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.008338715 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570292+0000", 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1756999999999998e-05 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572952+0000", 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026596889999999998 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44776 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2545919750000003, 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098382499999999998, 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44776 2024-09-16T22:16:59.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562092+0000", 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562094+0000", 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562101+0000", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.024e-06 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562115+0000", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4054e-05 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571783+0000", 2024-09-16T22:16:59.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096675010000000002 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571850+0000", 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.737e-05 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571930+0000", 2024-09-16T22:16:59.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9476999999999998e-05 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44777 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 77824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2545553389999995, 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099730849999999996, 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44777 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562129+0000", 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.51e-07 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562131+0000", 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562138+0000", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562144+0000", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049999999999998e-06 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571953+0000", 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098094210000000005 2024-09-16T22:16:59.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572022+0000", 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9190000000000007e-05 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572101+0000", 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8956999999999996e-05 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44778 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2545271860000007, 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010114365, 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44778 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562157+0000", 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562158+0000", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562165+0000", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562170+0000", 2024-09-16T22:16:59.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572125+0000", 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099544479999999994 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572193+0000", 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8262000000000005e-05 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572271+0000", 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7411999999999997e-05 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44779 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2545048960000003, 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010260511999999999, 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44779 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562179+0000", 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562185+0000", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6740000000000002e-06 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562191+0000", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9410000000000001e-06 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562197+0000", 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:16:59.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572297+0000", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010099542 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572361+0000", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4673999999999998e-05 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572439+0000", 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7717000000000001e-05 2024-09-16T22:16:59.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44780 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2544791800000006, 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01039753, 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44780 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562205+0000", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562207+0000", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.956e-06 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562213+0000", 2024-09-16T22:16:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5799999999999997e-06 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562218+0000", 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-06 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572461+0000", 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010243321999999999 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572526+0000", 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4292e-05 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572602+0000", 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5952999999999996e-05 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44781 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2544582759999994, 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010560442, 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44781 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562228+0000", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562234+0000", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562239+0000", 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0080000000000002e-06 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572626+0000", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010386799 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572693+0000", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7008999999999995e-05 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572786+0000", 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2346e-05 2024-09-16T22:16:59.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44782 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2544205500000007, 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010842269, 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44782 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9299999999999998e-07 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562266+0000", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562273+0000", 2024-09-16T22:16:59.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0670000000000003e-06 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562279+0000", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572810+0000", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010530684 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572944+0000", 2024-09-16T22:16:59.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134387 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573105+0000", 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000161096 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44783 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2543965369999999, 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011016781999999999, 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44783 2024-09-16T22:16:59.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:16:59.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562290+0000", 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2579999999999998e-06 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562296+0000", 2024-09-16T22:16:59.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562301+0000", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573160+0000", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010858945 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573221+0000", 2024-09-16T22:16:59.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0402000000000002e-05 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573304+0000", 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2807999999999997e-05 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:16:59.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:16:59.927 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:16:59.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:16:59.929 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:16:59.929+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:2779 write 0x9a48537 thru 0x9a53dd9 (0xb8a3 bytes) 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:2780 punch from 0x3ad3292 to 0x3ae28f2, (0xf660 bytes) 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:2781 write 0x6f4f4b thru 0x6fafbd (0x6073 bytes) 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:2782 writesame 0x514d5a7 thru 0x5153999 data_size 0x3(0x63f3 bytes) 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:2783 punch from 0xa77fef to 0xa7d93d, (0x594e bytes) 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi031.stdout:2788 write 0xaa4cb69 thru 0xaa4d482 (0x91a bytes) 2024-09-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi031.stdout:2790 read 0x77675c1 thru 0x776f8e3 (0x8323 bytes) 2024-09-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi031.stdout:2791 punch from 0x9c003ae to 0x9c0d648, (0xd29a bytes) 2024-09-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi031.stdout:2792 write 0x6e1c732 thru 0x6e1cea7 (0x776 bytes) 2024-09-16T22:16:59.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-16T22:16:59.933 INFO:teuthology.orchestra.run.smithi031.stdout:2793 read 0xa4704a thru 0xa4d4f2 (0x64a9 bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2794 write 0x5a04423 thru 0x5a06f1d (0x2afb bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2796 writesame 0xa6421f7 thru 0xa64d647 data_size 0x45(0xb451 bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2797 write 0xb0962a1 thru 0xb0a0f09 (0xac69 bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2799 read 0x221e7c0 thru 0x2228386 (0x9bc7 bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2800 writesame 0xd02fc6b thru 0xd03505c data_size 0x23(0x53f2 bytes) 2024-09-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi031.stdout:2803 trunc from 0xd03505d to 0xcfd07e 2024-09-16T22:16:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:2805 trunc from 0xcfd07e to 0x3f97ade 2024-09-16T22:16:59.972 INFO:teuthology.orchestra.run.smithi031.stdout:2806 writesame 0x618a585 thru 0x618e715 data_size 0xf(0x4191 bytes) 2024-09-16T22:16:59.972 INFO:teuthology.orchestra.run.smithi031.stdout:2808 write 0x46e5b0 thru 0x46ec26 (0x677 bytes) 2024-09-16T22:16:59.973 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:16:59.984 INFO:teuthology.orchestra.run.smithi031.stdout:2810 write 0x8506aaa thru 0x8512584 (0xbadb bytes) 2024-09-16T22:16:59.984 INFO:teuthology.orchestra.run.smithi031.stdout:2811 write 0xc4244 thru 0xca1cb (0x5f88 bytes) 2024-09-16T22:17:00.007 INFO:teuthology.orchestra.run.smithi031.stdout:2812 read 0x19a1f4b thru 0x19af5d8 (0xd68e bytes) 2024-09-16T22:17:00.007 INFO:teuthology.orchestra.run.smithi031.stdout:2813 read 0x5c8b6f6 thru 0x5c91320 (0x5c2b bytes) 2024-09-16T22:17:00.007 INFO:teuthology.orchestra.run.smithi031.stdout:2814 read 0x1367755 thru 0x1375977 (0xe223 bytes) 2024-09-16T22:17:00.008 INFO:teuthology.orchestra.run.smithi031.stdout:2815 read 0x7f95805 thru 0x7fa1e20 (0xc61c bytes) 2024-09-16T22:17:00.008 INFO:teuthology.orchestra.run.smithi031.stdout:2816 write 0xc1fd4 thru 0xcad6f (0x8d9c bytes) 2024-09-16T22:17:00.010 INFO:teuthology.orchestra.run.smithi031.stdout:2818 read 0x3aab2d3 thru 0x3ab9daf (0xeadd bytes) 2024-09-16T22:17:00.010 INFO:teuthology.orchestra.run.smithi031.stdout:2819 read 0x74125c7 thru 0x741b5ee (0x9028 bytes) 2024-09-16T22:17:00.010 INFO:teuthology.orchestra.run.smithi031.stdout:2820 punch from 0x214e83c to 0x21552ba, (0x6a7e bytes) 2024-09-16T22:17:00.010 INFO:teuthology.orchestra.run.smithi031.stdout:2821 read 0x4924528 thru 0x492cf87 (0x8a60 bytes) 2024-09-16T22:17:00.019 INFO:teuthology.orchestra.run.smithi031.stdout:2822 trunc from 0x8512585 to 0x6bbe88 2024-09-16T22:17:00.029 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:00.031+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:00.033 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.033 INFO:teuthology.orchestra.run.smithi031.stdout:2824 punch from 0x31b1c2 to 0x328a25, (0xd863 bytes) 2024-09-16T22:17:00.040 INFO:teuthology.orchestra.run.smithi031.stdout:2825 write 0x12f5d63 thru 0x1300dcb (0xb069 bytes) 2024-09-16T22:17:00.052 INFO:teuthology.orchestra.run.smithi031.stdout:2827 write 0x2d4a3e4 thru 0x2d52467 (0x8084 bytes) 2024-09-16T22:17:00.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.061 INFO:teuthology.orchestra.run.smithi031.stdout:2831 write 0x3720305 thru 0x3728315 (0x8011 bytes) 2024-09-16T22:17:00.080 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:00.081 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:00.081 INFO:teuthology.orchestra.run.smithi031.stdout:2832 write 0xec07f06 thru 0xec0d8c2 (0x59bd bytes) 2024-09-16T22:17:00.084 INFO:teuthology.orchestra.run.smithi031.stdout:2833 read 0x7b293cf thru 0x7b30bd0 (0x7802 bytes) 2024-09-16T22:17:00.084 INFO:teuthology.orchestra.run.smithi031.stdout:2834 writesame 0xabaf0cc thru 0xabbaa9d data_size 0x86(0xb9d2 bytes) 2024-09-16T22:17:00.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:00.102 INFO:teuthology.orchestra.run.smithi031.stdout:2835 trunc from 0xec0d8c3 to 0x1807168 2024-09-16T22:17:00.125 INFO:teuthology.orchestra.run.smithi031.stdout:2836 trunc from 0x1807168 to 0xe1979d3 2024-09-16T22:17:00.130 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:00.131+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:00.130 INFO:teuthology.orchestra.run.smithi031.stdout:2837 write 0x9b692a2 thru 0x9b70a94 (0x77f3 bytes) 2024-09-16T22:17:00.133 INFO:teuthology.orchestra.run.smithi031.stdout:2839 read 0x90baedb thru 0x90c6037 (0xb15d bytes) 2024-09-16T22:17:00.134 INFO:teuthology.orchestra.run.smithi031.stdout:2840 read 0x3bc7ea0 thru 0x3bc98b9 (0x1a1a bytes) 2024-09-16T22:17:00.134 INFO:teuthology.orchestra.run.smithi031.stdout:2841 write 0x2505bfc thru 0x2511df6 (0xc1fb bytes) 2024-09-16T22:17:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:2842 read 0xa1e9087 thru 0xa1e9fe1 (0xf5b bytes) 2024-09-16T22:17:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:2843 writesame 0x7906f54 thru 0x790c693 data_size 0x20(0x5740 bytes) 2024-09-16T22:17:00.142 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.142 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.143 INFO:teuthology.orchestra.run.smithi031.stdout:2847 read 0x71a4dca thru 0x71ac7e9 (0x7a20 bytes) 2024-09-16T22:17:00.143 INFO:teuthology.orchestra.run.smithi031.stdout:2848 write 0x11e86bf thru 0x11e8aa0 (0x3e2 bytes) 2024-09-16T22:17:00.144 INFO:teuthology.orchestra.run.smithi031.stdout:2849 writesame 0x3208afa thru 0x320cb13 data_size 0xf(0x401a bytes) 2024-09-16T22:17:00.147 INFO:teuthology.orchestra.run.smithi031.stdout:2852 punch from 0xc999b24 to 0xc9a8564, (0xea40 bytes) 2024-09-16T22:17:00.153 INFO:teuthology.orchestra.run.smithi031.stdout:2855 trunc from 0xe1979d3 to 0x8df31fc 2024-09-16T22:17:00.162 INFO:teuthology.orchestra.run.smithi031.stdout:2856 punch from 0x7d45c22 to 0x7d4f416, (0x97f4 bytes) 2024-09-16T22:17:00.162 INFO:teuthology.orchestra.run.smithi031.stdout:2857 punch from 0x6db53c7 to 0x6db9195, (0x3dce bytes) 2024-09-16T22:17:00.162 INFO:teuthology.orchestra.run.smithi031.stdout:2858 read 0x74a8ad0 thru 0x74aa618 (0x1b49 bytes) 2024-09-16T22:17:00.162 INFO:teuthology.orchestra.run.smithi031.stdout:2859 punch from 0x1df04d5 to 0x1dfdc16, (0xd741 bytes) 2024-09-16T22:17:00.170 INFO:teuthology.orchestra.run.smithi031.stdout:2860 writesame 0xb65f167 thru 0xb66742d data_size 0x1(0x82c7 bytes) 2024-09-16T22:17:00.170 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.170 INFO:teuthology.orchestra.run.smithi031.stdout:2862 write 0x995ccde thru 0x99623cb (0x56ee bytes) 2024-09-16T22:17:00.174 INFO:teuthology.orchestra.run.smithi031.stdout:2865 write 0x605a34b thru 0x6060168 (0x5e1e bytes) 2024-09-16T22:17:00.177 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.177 INFO:teuthology.orchestra.run.smithi031.stdout:2867 write 0x7297682 thru 0x7299590 (0x1f0f bytes) 2024-09-16T22:17:00.179 INFO:teuthology.orchestra.run.smithi031.stdout:2868 writesame 0x52299d4 thru 0x522fec2 data_size 0x3(0x64ef bytes) 2024-09-16T22:17:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:2871 read 0x9ff9361 thru 0xa005953 (0xc5f3 bytes) 2024-09-16T22:17:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:2872 writesame 0x8190459 thru 0x819d450 data_size 0x58(0xcff8 bytes) 2024-09-16T22:17:00.193 INFO:teuthology.orchestra.run.smithi031.stdout:2873 trunc from 0xb66742e to 0x704c7e7 2024-09-16T22:17:00.201 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:00.202 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:00.206 INFO:teuthology.orchestra.run.smithi031.stdout:2875 read 0x2b0c404 thru 0x2b0d67b (0x1278 bytes) 2024-09-16T22:17:00.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:00.213 INFO:teuthology.orchestra.run.smithi031.stdout:2876 trunc from 0x704c7e7 to 0xd115965 2024-09-16T22:17:00.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:00.219 INFO:teuthology.orchestra.run.smithi031.stdout:2879 punch from 0xab481bc to 0xab52769, (0xa5ad bytes) 2024-09-16T22:17:00.219 INFO:teuthology.orchestra.run.smithi031.stdout:2883 write 0xb3519cd thru 0xb35289f (0xed3 bytes) 2024-09-16T22:17:00.222 INFO:teuthology.orchestra.run.smithi031.stdout:2884 write 0x406fbc9 thru 0x4077edd (0x8315 bytes) 2024-09-16T22:17:00.226 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.228 INFO:teuthology.orchestra.run.smithi031.stdout:2888 trunc from 0xd115965 to 0xb1ab726 2024-09-16T22:17:00.245 INFO:teuthology.orchestra.run.smithi031.stdout:2889 trunc from 0xb1ab726 to 0x3043d61 2024-09-16T22:17:00.255 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:00.257+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:00.257 INFO:teuthology.orchestra.run.smithi031.stdout:2891 write 0x145b829 thru 0x14677fa (0xbfd2 bytes) 2024-09-16T22:17:00.266 INFO:teuthology.orchestra.run.smithi031.stdout:2892 writesame 0x3daeaa9 thru 0x3daf79f data_size 0x1(0xcf7 bytes) 2024-09-16T22:17:00.266 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.266 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.277 INFO:teuthology.orchestra.run.smithi031.stdout:2895 write 0x7e84205 thru 0x7e92ac1 (0xe8bd bytes) 2024-09-16T22:17:00.280 INFO:teuthology.orchestra.run.smithi031.stdout:2896 read 0x50e149e thru 0x50e31ad (0x1d10 bytes) 2024-09-16T22:17:00.280 INFO:teuthology.orchestra.run.smithi031.stdout:2897 writesame 0x3051efe thru 0x305dbf8 data_size 0x1(0xbcfb bytes) 2024-09-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:00.318 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40648 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428597088, 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099836779999999993, 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40648 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890095+0000", 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890102+0000", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1679999999999998e-06 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890111+0000", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.242e-06 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.899922+0000", 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098110300000000001 2024-09-16T22:17:00.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.899992+0000", 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0154999999999995e-05 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900077+0000", 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4709000000000005e-05 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40649 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428572154999999, 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010144003, 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40649 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:00.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890120+0000", 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890127+0000", 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0679999999999999e-06 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890132+0000", 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900103+0000", 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099713340000000001 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900174+0000", 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0463000000000006e-05 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900262+0000", 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7903000000000002e-05 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40650 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.42855123, 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010302911, 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40650 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890141+0000", 2024-09-16T22:17:00.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8190000000000001e-06 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890148+0000", 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890156+0000", 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.619e-06 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900295+0000", 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010139005 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900367+0000", 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1810000000000005e-05 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900442+0000", 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4510000000000003e-05 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40651 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428526835, 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010449882000000001, 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40651 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890164+0000", 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890166+0000", 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.996e-06 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890173+0000", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9990000000000002e-06 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890177+0000", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9099999999999996e-06 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900469+0000", 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010291873999999999 2024-09-16T22:17:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900539+0000", 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9448000000000003e-05 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900613+0000", 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4260999999999999e-05 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40652 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428506077, 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01060757, 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40652 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890186+0000", 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9199999999999998e-06 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890192+0000", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8889999999999999e-06 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890196+0000", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900642+0000", 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010445217 2024-09-16T22:17:00.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900712+0000", 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0009e-05 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900791+0000", 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9779999999999998e-05 2024-09-16T22:17:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40653 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428485421, 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010775066999999999, 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40653 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890212+0000", 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.759e-06 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890219+0000", 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3379999999999999e-06 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890224+0000", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0860000000000001e-06 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900826+0000", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010601522 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900898+0000", 2024-09-16T22:17:00.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1967999999999999e-05 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900980+0000", 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2025000000000002e-05 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40654 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428459115000001, 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010923034, 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40654 2024-09-16T22:17:00.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890233+0000", 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890240+0000", 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.922e-06 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890245+0000", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8520000000000003e-06 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901007+0000", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010761517999999999 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901074+0000", 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901154+0000", 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9721999999999993e-05 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40655 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428438609000001, 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011076674, 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.354 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40655 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890252+0000", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-07 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890253+0000", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890259+0000", 2024-09-16T22:17:00.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6890000000000001e-06 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890263+0000", 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-06 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901180+0000", 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010916762 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901243+0000", 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2841000000000004e-05 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901328+0000", 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5346000000000006e-05 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40656 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 120832~1115] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428420744, 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011320360999999999, 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40656 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890270+0000", 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890271+0000", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7999999999999999e-06 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890278+0000", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890283+0000", 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7380000000000002e-06 2024-09-16T22:17:00.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901365+0000", 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011082277 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901476+0000", 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111156 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901590+0000", 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113634 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40657 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 129024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.428401266, 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011486682999999999, 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:00.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40657 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:00.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890298+0000", 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2199999999999998e-06 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890309+0000", 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0562e-05 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890314+0000", 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901629+0000", 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011315218 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901707+0000", 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7736000000000002e-05 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901775+0000", 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8682000000000004e-05 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42587 3.8 e51/23 3:1432c3a7:::rbd_data.11cd49ca855d.0000000000000007:head v 51'187, mlcod=51'185)", 2024-09-16T22:17:00.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.136650356000001, 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025255219999999998, 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182041+0000", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8679999999999999e-06 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.184972+0000", 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029307539999999998 2024-09-16T22:17:00.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.184994+0000", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560999999999999e-05 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185003+0000", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3060000000000003e-06 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185066+0000", 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3235e-05 2024-09-16T22:17:00.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185092+0000", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207246+0000", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022153889 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207295+0000", 2024-09-16T22:17:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8668000000000002e-05 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42734 3.b 3:d3231630:::rbd_data.11cd49ca855d.000000000000000e:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.982012451999999, 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020378594, 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42734 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336679+0000", 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6270000000000001e-06 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336686+0000", 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4980000000000003e-06 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336698+0000", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2758000000000001e-05 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336705+0000", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6209999999999998e-06 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336766+0000", 2024-09-16T22:17:00.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0659999999999999e-05 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336771+0000", 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8130000000000003e-06 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.338051+0000", 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001280719 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.338092+0000", 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1088999999999999e-05 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.340820+0000", 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027281269999999999 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:00.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.356965+0000", 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016144807000000001 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.356977+0000", 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1474999999999999e-05 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.357009+0000", 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2561999999999998e-05 2024-09-16T22:17:00.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.357056+0000", 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6838000000000001e-05 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44290 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7687731469999992, 2024-09-16T22:17:00.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099311009999999995, 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44290 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549919+0000", 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5400000000000001e-06 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549924+0000", 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5849999999999999e-06 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549929+0000", 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5560000000000001e-06 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559836+0000", 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009907585 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559848+0000", 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1522000000000001e-05 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44291 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:00.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7687559969999995, 2024-09-16T22:17:00.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099459179999999998, 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44291 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.15e-07 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549936+0000", 2024-09-16T22:17:00.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-06 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549941+0000", 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.614e-06 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549946+0000", 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4499999999999997e-06 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559871+0000", 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099246149999999995 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559880+0000", 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3989999999999993e-06 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44292 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7687389119999999, 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099693360000000005, 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44292 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3799999999999998e-07 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549953+0000", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.502e-06 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549959+0000", 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549963+0000", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7389999999999999e-06 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559912+0000", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099484870000000006 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559920+0000", 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5299999999999996e-06 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44293 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7687071979999995, 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099827490000000008, 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44293 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549984+0000", 2024-09-16T22:17:00.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6600000000000005e-07 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549988+0000", 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0210000000000001e-06 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549996+0000", 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4479999999999994e-06 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550003+0000", 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6309999999999997e-06 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559954+0000", 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099515709999999993 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559966+0000", 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1212e-05 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44294 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7686753639999999, 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099995169999999994, 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44294 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550018+0000", 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4959999999999999e-06 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550025+0000", 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3139999999999998e-06 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550030+0000", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5589999999999996e-06 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560002+0000", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099713679999999996 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560014+0000", 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2296000000000001e-05 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44295 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.7686521600000003, 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010122055, 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:00.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44295 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550048+0000", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8579999999999995e-06 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550056+0000", 2024-09-16T22:17:00.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4309999999999997e-06 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550061+0000", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7679999999999997e-06 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560151+0000", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010089677999999999 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560160+0000", 2024-09-16T22:17:00.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8519999999999993e-06 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:59483 3.c e60/23 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head v 60'175, mlcod=59'173)", 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3791689499999999, 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018712162000000001, 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:00.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:00.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4900000000000002e-07 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946524+0000", 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070025319999999997 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946548+0000", 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4498000000000002e-05 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946560+0000", 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1539000000000001e-05 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946637+0000", 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6970000000000003e-05 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946662+0000", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5369000000000001e-05 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958202+0000", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011539667999999999 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958233+0000", 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1337000000000002e-05 2024-09-16T22:17:00.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:59484 3.c e60/23 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head v 60'177, mlcod=59'173)", 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3709382720000001, 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010812252, 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:00.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:00.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6000000000000003e-07 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947758+0000", 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999998e-06 2024-09-16T22:17:00.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947768+0000", 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0094e-05 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947774+0000", 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3130000000000004e-06 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:00.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950861+0000", 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030862279999999999 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950876+0000", 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5722e-05 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958535+0000", 2024-09-16T22:17:00.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076586629999999996 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958564+0000", 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9051999999999999e-05 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:00.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:00.420 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:00.422 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:00.423+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:00.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2900 writesame 0xe936895 thru 0xe93d1f9 data_size 0x1(0x6965 bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2901 punch from 0xb61020b to 0xb615627, (0x541c bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2903 write 0x40b99d7 thru 0x40bc305 (0x292f bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2904 trunc from 0xe93d1fa to 0xa22ccc6 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2907 writesame 0x11cedfe thru 0x11d3f14 data_size 0x1(0x5117 bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2910 write 0x416b808 thru 0x417363a (0x7e33 bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2911 writesame 0xc808e05 thru 0xc80f753 data_size 0x1(0x694f bytes) 2024-09-16T22:17:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:2912 read 0x5582a3 thru 0x55f0d7 (0x6e35 bytes) 2024-09-16T22:17:00.425 INFO:teuthology.orchestra.run.smithi031.stdout:2913 read 0x764ae3e thru 0x764ecad (0x3e70 bytes) 2024-09-16T22:17:00.425 INFO:teuthology.orchestra.run.smithi031.stdout:2914 write 0x846224a thru 0x846397c (0x1733 bytes) 2024-09-16T22:17:00.425 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:00.425 INFO:teuthology.orchestra.run.smithi031.stdout:2915 trunc from 0xc80f754 to 0x645a8a 2024-09-16T22:17:00.426 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:00.426 INFO:teuthology.orchestra.run.smithi031.stdout:2918 punch from 0x281b7 to 0x2977a, (0x15c3 bytes) 2024-09-16T22:17:00.427 INFO:teuthology.orchestra.run.smithi031.stdout:2919 trunc from 0x645a8a to 0x8a50b4e 2024-09-16T22:17:00.427 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:00.428 INFO:teuthology.orchestra.run.smithi031.stdout:2921 writesame 0x9766795 thru 0x9770921 data_size 0x1(0xa18d bytes) 2024-09-16T22:17:00.428 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.428 INFO:teuthology.orchestra.run.smithi031.stdout:2924 trunc from 0x9770922 to 0x12912a2 2024-09-16T22:17:00.430 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:00.441 INFO:teuthology.orchestra.run.smithi031.stdout:2925 write 0x4e32131 thru 0x4e3e5bc (0xc48c bytes) 2024-09-16T22:17:00.445 INFO:teuthology.orchestra.run.smithi031.stdout:2926 read 0x49b470 thru 0x49f02e (0x3bbf bytes) 2024-09-16T22:17:00.445 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:00.446 INFO:teuthology.orchestra.run.smithi031.stdout:2929 write 0x2c62a3c thru 0x2c6c230 (0x97f5 bytes) 2024-09-16T22:17:00.449 INFO:teuthology.orchestra.run.smithi031.stdout:2930 punch from 0x25c3f0b to 0x25c47d5, (0x8ca bytes) 2024-09-16T22:17:00.449 INFO:teuthology.orchestra.run.smithi031.stdout:2931 read 0x152ef23 thru 0x1535f84 (0x7062 bytes) 2024-09-16T22:17:00.449 INFO:teuthology.orchestra.run.smithi031.stdout:2932 clone 27 order 20 su 32768 sc 8 2024-09-16T22:17:00.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:00.468 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:00.469 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:00.485 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:00.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:00.539 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:00.540 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:00.540 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:00.541+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:00.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:00.623 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:00.624+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:00.696 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:00.696 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:00.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:00.723 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:00.725+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:00.811 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:00.811 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:00.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:00.864 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:00.866+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:00.925 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:00.926+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:00.927 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:00.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:00.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:01.026 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:01.028+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:01.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:01.135 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:01.137+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:01.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:17:01.208 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone26 from 0x4e3e5bd (overlap 0x645a8a) to 0x282437 2024-09-16T22:17:01.211 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.211 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:01.264 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T22:17:01.264 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:01.266+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:01.329 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:01.330+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:01.350 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.350 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:01.430 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:01.432+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:01.514 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.514 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:01.533 INFO:teuthology.orchestra.run.smithi031.stdout:2933 read 0x42861d6 thru 0x428ddbd (0x7be8 bytes) 2024-09-16T22:17:01.533 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:01.535+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:01.533 INFO:teuthology.orchestra.run.smithi031.stdout:2934 read 0x46f8969 thru 0x46ff826 (0x6ebe bytes) 2024-09-16T22:17:01.534 INFO:teuthology.orchestra.run.smithi031.stdout:2936 writesame 0x3506b3 thru 0x35eb42 data_size 0x6a(0xe490 bytes) 2024-09-16T22:17:01.552 INFO:teuthology.orchestra.run.smithi031.stdout:2937 punch from 0x48e7791 to 0x48f45aa, (0xce19 bytes) 2024-09-16T22:17:01.552 INFO:teuthology.orchestra.run.smithi031.stdout:2939 read 0x3887918 thru 0x3893d52 (0xc43b bytes) 2024-09-16T22:17:01.552 INFO:teuthology.orchestra.run.smithi031.stdout:2940 writesame 0x36cb675 thru 0x36d4317 data_size 0xb(0x8ca3 bytes) 2024-09-16T22:17:01.565 INFO:teuthology.orchestra.run.smithi031.stdout:2941 trunc from 0x4e3e5bd to 0xaa39854 2024-09-16T22:17:01.575 INFO:teuthology.orchestra.run.smithi031.stdout:2942 writesame 0xaf844f4 thru 0xaf860f5 data_size 0xf(0x1c02 bytes) 2024-09-16T22:17:01.586 INFO:teuthology.orchestra.run.smithi031.stdout:2943 trunc from 0xaf860f6 to 0x138275c 2024-09-16T22:17:01.615 INFO:teuthology.orchestra.run.smithi031.stdout:2944 punch from 0x2f4236 to 0x2fb9ea, (0x77b4 bytes) 2024-09-16T22:17:01.630 INFO:teuthology.orchestra.run.smithi031.stdout:2945 write 0xa92abaa thru 0xa92de54 (0x32ab bytes) 2024-09-16T22:17:01.632 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:01.634+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:01.633 INFO:teuthology.orchestra.run.smithi031.stdout:2946 write 0x27bf92 thru 0x27ef6d (0x2fdc bytes) 2024-09-16T22:17:01.642 INFO:teuthology.orchestra.run.smithi031.stdout:2948 write 0x7ea0760 thru 0x7ea67e5 (0x6086 bytes) 2024-09-16T22:17:01.645 INFO:teuthology.orchestra.run.smithi031.stdout:2950 write 0x1ef762a thru 0x1efc456 (0x4e2d bytes) 2024-09-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi031.stdout:2951 punch from 0x60bfb0 to 0x617e46, (0xbe96 bytes) 2024-09-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi031.stdout:2953 write 0x752f940 thru 0x7532b79 (0x323a bytes) 2024-09-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.656 INFO:teuthology.orchestra.run.smithi031.stdout:2954 trunc from 0xa92de55 to 0xd23313a 2024-09-16T22:17:01.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:01.662 INFO:teuthology.orchestra.run.smithi031.stdout:2956 read 0xbdf87e9 thru 0xbdfef90 (0x67a8 bytes) 2024-09-16T22:17:01.662 INFO:teuthology.orchestra.run.smithi031.stdout:2957 trunc from 0xd23313a to 0xd007a35 2024-09-16T22:17:01.675 INFO:teuthology.orchestra.run.smithi031.stdout:2958 trunc from 0xd007a35 to 0x5363026 2024-09-16T22:17:01.684 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:01.684 INFO:teuthology.orchestra.run.smithi031.stdout:2960 read 0x44c7793 thru 0x44d3f79 (0xc7e7 bytes) 2024-09-16T22:17:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:2961 writesame 0x8e8b536 thru 0x8e8f785 data_size 0x8(0x4250 bytes) 2024-09-16T22:17:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:2962 write 0x2421417 thru 0x242f14c (0xdd36 bytes) 2024-09-16T22:17:01.697 INFO:teuthology.orchestra.run.smithi031.stdout:2964 read 0xbe1843 thru 0xbe4770 (0x2f2e bytes) 2024-09-16T22:17:01.707 INFO:teuthology.orchestra.run.smithi031.stdout:2965 trunc from 0x8e8f786 to 0xb03209 2024-09-16T22:17:01.733 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:01.735+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:01.748 INFO:teuthology.orchestra.run.smithi031.stdout:2966 write 0xa0f15ed thru 0xa0f8667 (0x707b bytes) 2024-09-16T22:17:01.752 INFO:teuthology.orchestra.run.smithi031.stdout:2968 write 0x297f72d thru 0x298f6bd (0xff91 bytes) 2024-09-16T22:17:01.757 INFO:teuthology.orchestra.run.smithi031.stdout:2969 read 0x363e377 thru 0x364276d (0x43f7 bytes) 2024-09-16T22:17:01.764 INFO:teuthology.orchestra.run.smithi031.stdout:2970 writesame 0xac2f2f2 thru 0xac34b72 data_size 0x1(0x5881 bytes) 2024-09-16T22:17:01.766 INFO:teuthology.orchestra.run.smithi031.stdout:2971 trunc from 0xac34b73 to 0xd0a4d28 2024-09-16T22:17:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:2972 read 0x316d2d0 thru 0x317447c (0x71ad bytes) 2024-09-16T22:17:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:2973 write 0x495bc79 thru 0x4960b8c (0x4f14 bytes) 2024-09-16T22:17:01.779 INFO:teuthology.orchestra.run.smithi031.stdout:2974 write 0xc230549 thru 0xc240088 (0xfb40 bytes) 2024-09-16T22:17:01.787 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.787 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.789 INFO:teuthology.orchestra.run.smithi031.stdout:2975 read 0xab33245 thru 0xab3a915 (0x76d1 bytes) 2024-09-16T22:17:01.789 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:01.789 INFO:teuthology.orchestra.run.smithi031.stdout:2977 read 0x4ee73be thru 0x4ef636b (0xefae bytes) 2024-09-16T22:17:01.790 INFO:teuthology.orchestra.run.smithi031.stdout:2978 read 0xa84fc0a thru 0xa85a54b (0xa942 bytes) 2024-09-16T22:17:01.790 INFO:teuthology.orchestra.run.smithi031.stdout:2980 write 0x8f74a56 thru 0x8f845f8 (0xfba3 bytes) 2024-09-16T22:17:01.796 INFO:teuthology.orchestra.run.smithi031.stdout:2981 read 0x1bcf14d thru 0x1bdb891 (0xc745 bytes) 2024-09-16T22:17:01.796 INFO:teuthology.orchestra.run.smithi031.stdout:2982 write 0x21d6c79 thru 0x21db5a3 (0x492b bytes) 2024-09-16T22:17:01.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:01.806 INFO:teuthology.orchestra.run.smithi031.stdout:2983 trunc from 0xd0a4d28 to 0xa7f7a5d 2024-09-16T22:17:01.819 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:01.819 INFO:teuthology.orchestra.run.smithi031.stdout:2986 read 0x93ed102 thru 0x93f5b93 (0x8a92 bytes) 2024-09-16T22:17:01.819 INFO:teuthology.orchestra.run.smithi031.stdout:2988 read 0x4e07896 thru 0x4e17077 (0xf7e2 bytes) 2024-09-16T22:17:01.824 INFO:teuthology.orchestra.run.smithi031.stdout:2989 write 0xaf5f69d thru 0xaf69cf7 (0xa65b bytes) 2024-09-16T22:17:01.832 INFO:teuthology.orchestra.run.smithi031.stdout:2990 writesame 0x54696db thru 0x546ece9 data_size 0x1(0x560f bytes) 2024-09-16T22:17:01.834 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:01.836+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:01.845 INFO:teuthology.orchestra.run.smithi031.stdout:2993 read 0x52ca116 thru 0x52cd688 (0x3573 bytes) 2024-09-16T22:17:01.845 INFO:teuthology.orchestra.run.smithi031.stdout:2994 read 0x38a454c thru 0x38b3de8 (0xf89d bytes) 2024-09-16T22:17:01.845 INFO:teuthology.orchestra.run.smithi031.stdout:2995 punch from 0x316949d to 0x316f237, (0x5d9a bytes) 2024-09-16T22:17:01.845 INFO:teuthology.orchestra.run.smithi031.stdout:2997 punch from 0x9104fcb to 0x9114499, (0xf4ce bytes) 2024-09-16T22:17:01.845 INFO:teuthology.orchestra.run.smithi031.stdout:2998 writesame 0x8b14544 thru 0x8b185b8 data_size 0x1(0x4075 bytes) 2024-09-16T22:17:01.860 INFO:teuthology.orchestra.run.smithi031.stdout:3000 trunc from 0xaf69cf8 to 0xe1cf8df 2024-09-16T22:17:01.865 INFO:teuthology.orchestra.run.smithi031.stdout:3001 punch from 0xbc04301 to 0xbc07cbc, (0x39bb bytes) 2024-09-16T22:17:01.865 INFO:teuthology.orchestra.run.smithi031.stdout:3002 writesame 0x8955d34 thru 0x895d993 data_size 0x50(0x7c60 bytes) 2024-09-16T22:17:01.867 INFO:teuthology.orchestra.run.smithi031.stdout:3004 punch from 0xa4dcbbe to 0xa4eab61, (0xdfa3 bytes) 2024-09-16T22:17:01.867 INFO:teuthology.orchestra.run.smithi031.stdout:3005 write 0x6a5f80c thru 0x6a673a9 (0x7b9e bytes) 2024-09-16T22:17:01.872 INFO:teuthology.orchestra.run.smithi031.stdout:3006 read 0x954a540 thru 0x9553395 (0x8e56 bytes) 2024-09-16T22:17:01.872 INFO:teuthology.orchestra.run.smithi031.stdout:3007 write 0x2840f0 thru 0x2902d9 (0xc1ea bytes) 2024-09-16T22:17:01.885 INFO:teuthology.orchestra.run.smithi031.stdout:3009 read 0x63b33ef thru 0x63bbb7c (0x878e bytes) 2024-09-16T22:17:01.886 INFO:teuthology.orchestra.run.smithi031.stdout:3010 trunc from 0xe1cf8df to 0xd24dcce 2024-09-16T22:17:01.892 INFO:teuthology.orchestra.run.smithi031.stdout:3012 writesame 0x892579b thru 0x892db84 data_size 0x16(0x83ea bytes) 2024-09-16T22:17:01.896 INFO:teuthology.orchestra.run.smithi031.stdout:3013 punch from 0x5e33304 to 0x5e39bb0, (0x68ac bytes) 2024-09-16T22:17:01.896 INFO:teuthology.orchestra.run.smithi031.stdout:3015 read 0x92d3912 thru 0x92e150a (0xdbf9 bytes) 2024-09-16T22:17:01.902 INFO:teuthology.orchestra.run.smithi031.stdout:3016 write 0xe0cecf8 thru 0xe0d820d (0x9516 bytes) 2024-09-16T22:17:01.907 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:01.908 INFO:teuthology.orchestra.run.smithi031.stdout:3018 punch from 0x99887bd to 0x9990ebd, (0x8700 bytes) 2024-09-16T22:17:01.908 INFO:teuthology.orchestra.run.smithi031.stdout:3020 read 0x2d5c6e7 thru 0x2d5e2a0 (0x1bba bytes) 2024-09-16T22:17:01.908 INFO:teuthology.orchestra.run.smithi031.stdout:3021 writesame 0xb486db thru 0xb57dfd data_size 0x1(0xf723 bytes) 2024-09-16T22:17:01.926 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:01.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:01.930 INFO:teuthology.orchestra.run.smithi031.stdout:3025 write 0x451d4df thru 0x4526db7 (0x98d9 bytes) 2024-09-16T22:17:01.934 INFO:teuthology.orchestra.run.smithi031.stdout:3027 write 0x8904ab0 thru 0x891431d (0xf86e bytes) 2024-09-16T22:17:01.935 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:01.936+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:01.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:3028 punch from 0xc14dac8 to 0xc1509e4, (0x2f1c bytes) 2024-09-16T22:17:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:3030 write 0x4e53293 thru 0x4e6237c (0xf0ea bytes) 2024-09-16T22:17:01.945 INFO:teuthology.orchestra.run.smithi031.stdout:3031 write 0xe28707c thru 0xe288e7c (0x1e01 bytes) 2024-09-16T22:17:01.948 INFO:teuthology.orchestra.run.smithi031.stdout:3032 write 0xddd93f5 thru 0xdde41bc (0xadc8 bytes) 2024-09-16T22:17:01.952 INFO:teuthology.orchestra.run.smithi031.stdout:3033 write 0x6d1f175 thru 0x6d2ce73 (0xdcff bytes) 2024-09-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi031.stdout:3034 write 0x28e26fb thru 0x28eef2e (0xc834 bytes) 2024-09-16T22:17:01.958 INFO:teuthology.orchestra.run.smithi031.stdout:3035 write 0xc37e20d thru 0xc38abf7 (0xc9eb bytes) 2024-09-16T22:17:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:3036 write 0xc482dbe thru 0xc488d2c (0x5f6f bytes) 2024-09-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi031.stdout:3037 writesame 0xd234e34 thru 0xd236e2d data_size 0x1(0x1ffa bytes) 2024-09-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi031.stdout:3039 punch from 0xd58a166 to 0xd591a0c, (0x78a6 bytes) 2024-09-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi031.stdout:3041 punch from 0x7a24558 to 0x7a273f1, (0x2e99 bytes) 2024-09-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi031.stdout:3042 writesame 0x965622a thru 0x9665f87 data_size 0x52(0xfd5e bytes) 2024-09-16T22:17:01.979 INFO:teuthology.orchestra.run.smithi031.stdout:3043 read 0x337933f thru 0x338816a (0xee2c bytes) 2024-09-16T22:17:01.979 INFO:teuthology.orchestra.run.smithi031.stdout:3044 read 0xc0c7bf5 thru 0xc0d42b8 (0xc6c4 bytes) 2024-09-16T22:17:01.980 INFO:teuthology.orchestra.run.smithi031.stdout:3046 write 0x71ee548 thru 0x71f8a26 (0xa4df bytes) 2024-09-16T22:17:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:3047 read 0xc0678e thru 0xc166ae (0xff21 bytes) 2024-09-16T22:17:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:3048 read 0x148e4a6 thru 0x14978c5 (0x9420 bytes) 2024-09-16T22:17:01.987 INFO:teuthology.orchestra.run.smithi031.stdout:3051 trunc from 0xe288e7d to 0xd7798ad 2024-09-16T22:17:01.999 INFO:teuthology.orchestra.run.smithi031.stdout:3052 punch from 0x801243b to 0x801e7ad, (0xc372 bytes) 2024-09-16T22:17:01.999 INFO:teuthology.orchestra.run.smithi031.stdout:3054 writesame 0x4b65f9f thru 0x4b715b1 data_size 0x3(0xb613 bytes) 2024-09-16T22:17:02.006 INFO:teuthology.orchestra.run.smithi031.stdout:3056 writesame 0x232f58f thru 0x233a441 data_size 0x1(0xaeb3 bytes) 2024-09-16T22:17:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:3057 read 0x6d08d5b thru 0x6d0d917 (0x4bbd bytes) 2024-09-16T22:17:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:3058 write 0x724bf3f thru 0x72593a5 (0xd467 bytes) 2024-09-16T22:17:02.033 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.033 INFO:teuthology.orchestra.run.smithi031.stdout:3064 write 0xce90474 thru 0xce955d4 (0x5161 bytes) 2024-09-16T22:17:02.035 INFO:teuthology.orchestra.run.smithi031.stdout:3066 read 0x58795d7 thru 0x587aec9 (0x18f3 bytes) 2024-09-16T22:17:02.036 INFO:teuthology.orchestra.run.smithi031.stdout:3067 read 0x27e1bcd thru 0x27e2623 (0xa57 bytes) 2024-09-16T22:17:02.036 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.036 INFO:teuthology.orchestra.run.smithi031.stdout:3069 writesame 0x133a752 thru 0x13475fe data_size 0x1(0xcead bytes) 2024-09-16T22:17:02.036 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:02.037+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:02.052 INFO:teuthology.orchestra.run.smithi031.stdout:3070 write 0x2ac39ed thru 0x2ac3a40 (0x54 bytes) 2024-09-16T22:17:02.053 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.053 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.060 INFO:teuthology.orchestra.run.smithi031.stdout:3071 write 0xe42516e thru 0xe433afa (0xe98d bytes) 2024-09-16T22:17:02.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:02.074 INFO:teuthology.orchestra.run.smithi031.stdout:3072 trunc from 0xe433afb to 0x97bc221 2024-09-16T22:17:02.088 INFO:teuthology.orchestra.run.smithi031.stdout:3073 trunc from 0x97bc221 to 0xa26e7a4 2024-09-16T22:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout:3075 read 0x7545952 thru 0x7552e55 (0xd504 bytes) 2024-09-16T22:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout:3077 writesame 0x29149f0 thru 0x2918240 data_size 0x1(0x3851 bytes) 2024-09-16T22:17:02.108 INFO:teuthology.orchestra.run.smithi031.stdout:3078 write 0xb552649 thru 0xb554665 (0x201d bytes) 2024-09-16T22:17:02.111 INFO:teuthology.orchestra.run.smithi031.stdout:3079 punch from 0x46fbf0a to 0x46fe8d7, (0x29cd bytes) 2024-09-16T22:17:02.112 INFO:teuthology.orchestra.run.smithi031.stdout:3080 write 0x46b9e6 thru 0x47ad99 (0xf3b4 bytes) 2024-09-16T22:17:02.124 INFO:teuthology.orchestra.run.smithi031.stdout:3081 punch from 0x1a970c3 to 0x1aa0664, (0x95a1 bytes) 2024-09-16T22:17:02.131 INFO:teuthology.orchestra.run.smithi031.stdout:3084 writesame 0xcaf81b6 thru 0xcafd15d data_size 0x4(0x4fa8 bytes) 2024-09-16T22:17:02.131 INFO:teuthology.orchestra.run.smithi031.stdout:3085 writesame 0x8981a94 thru 0x89901f1 data_size 0x5(0xe75e bytes) 2024-09-16T22:17:02.136 INFO:teuthology.orchestra.run.smithi031.stdout:3086 read 0xa7e9246 thru 0xa7f283c (0x95f7 bytes) 2024-09-16T22:17:02.136 INFO:teuthology.orchestra.run.smithi031.stdout:3087 writesame 0x18ce048 thru 0x18d15ea data_size 0x17(0x35a3 bytes) 2024-09-16T22:17:02.136 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:02.138+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:02.140 INFO:teuthology.orchestra.run.smithi031.stdout:3088 read 0x9d32e87 thru 0x9d3a308 (0x7482 bytes) 2024-09-16T22:17:02.140 INFO:teuthology.orchestra.run.smithi031.stdout:3091 punch from 0x2ea0122 to 0x2eae134, (0xe012 bytes) 2024-09-16T22:17:02.140 INFO:teuthology.orchestra.run.smithi031.stdout:3092 read 0x91b329a thru 0x91b9e30 (0x6b97 bytes) 2024-09-16T22:17:02.140 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.140 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:3095 read 0x23c90f2 thru 0x23d6241 (0xd150 bytes) 2024-09-16T22:17:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:3097 read 0x818e48 thru 0x81bd6f (0x2f28 bytes) 2024-09-16T22:17:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:3099 read 0x628ffcc thru 0x629fdc9 (0xfdfe bytes) 2024-09-16T22:17:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:3101 read 0xa8509f0 thru 0xa85c682 (0xbc93 bytes) 2024-09-16T22:17:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:3102 write 0xa24eaaf thru 0xa252f48 (0x449a bytes) 2024-09-16T22:17:02.145 INFO:teuthology.orchestra.run.smithi031.stdout:3103 write 0xad54ccc thru 0xad61adf (0xce14 bytes) 2024-09-16T22:17:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:17:02.166 INFO:teuthology.orchestra.run.smithi031.stdout:3104 trunc from 0xcafd15e to 0x15f28a 2024-09-16T22:17:02.185 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.185 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:02.206 INFO:teuthology.orchestra.run.smithi031.stdout:3105 trunc from 0x15f28a to 0x66ebd2e 2024-09-16T22:17:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:3110 read 0x5320b9 thru 0x532354 (0x29c bytes) 2024-09-16T22:17:02.212 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.212 INFO:teuthology.orchestra.run.smithi031.stdout:3112 read 0x3540414 thru 0x354b7b6 (0xb3a3 bytes) 2024-09-16T22:17:02.212 INFO:teuthology.orchestra.run.smithi031.stdout:3118 write 0x2494790 thru 0x249e3fd (0x9c6e bytes) 2024-09-16T22:17:02.219 INFO:teuthology.orchestra.run.smithi031.stdout:3119 write 0x55bf2ab thru 0x55c97d9 (0xa52f bytes) 2024-09-16T22:17:02.227 INFO:teuthology.orchestra.run.smithi031.stdout:3120 write 0x18c53bb thru 0x18cc7ac (0x73f2 bytes) 2024-09-16T22:17:02.237 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:02.239+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:02.244 INFO:teuthology.orchestra.run.smithi031.stdout:3122 write 0xbac2984 thru 0xbac8ad4 (0x6151 bytes) 2024-09-16T22:17:02.251 INFO:teuthology.orchestra.run.smithi031.stdout:3123 punch from 0x4978ce4 to 0x4988149, (0xf465 bytes) 2024-09-16T22:17:02.251 INFO:teuthology.orchestra.run.smithi031.stdout:3124 write 0x3062b03 thru 0x30688a5 (0x5da3 bytes) 2024-09-16T22:17:02.256 INFO:teuthology.orchestra.run.smithi031.stdout:3125 punch from 0x6ee22ae to 0x6ee6927, (0x4679 bytes) 2024-09-16T22:17:02.256 INFO:teuthology.orchestra.run.smithi031.stdout:3126 read 0x1f246a thru 0x20156b (0xf102 bytes) 2024-09-16T22:17:02.257 INFO:teuthology.orchestra.run.smithi031.stdout:3127 write 0x3b15473 thru 0x3b190c3 (0x3c51 bytes) 2024-09-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi031.stdout:3129 trunc from 0xbac8ad5 to 0x943d06a 2024-09-16T22:17:02.275 INFO:teuthology.orchestra.run.smithi031.stdout:3130 trunc from 0x943d06a to 0xbe97fe6 2024-09-16T22:17:02.281 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:02.281 INFO:teuthology.orchestra.run.smithi031.stdout:3132 read 0x55e1887 thru 0x55e5b46 (0x42c0 bytes) 2024-09-16T22:17:02.281 INFO:teuthology.orchestra.run.smithi031.stdout:3136 clone 28 order 24 su 4194304 sc 12 2024-09-16T22:17:02.304 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.304 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:02.337 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:02.339+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:02.439 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:02.440+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:02.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:02.530 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.530 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:02.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:02.560+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:02.640 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:02.641 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:02.642+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:02.642 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:02.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:02.657 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:02.657 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:02.674 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:02.674 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:02.690 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:02.706 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:02.707 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:02.723 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:02.739 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:02.739 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:02.741 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:02.743+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:02.755 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18014 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256837411999999, 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026763540999999998, 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18014 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500571+0000", 2024-09-16T22:17:02.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500572+0000", 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.085e-06 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500576+0000", 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8229999999999997e-06 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500578+0000", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3360000000000002e-06 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527256+0000", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026677421 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527297+0000", 2024-09-16T22:17:02.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1028000000000001e-05 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527334+0000", 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7469000000000003e-05 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18015 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14123008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256827027, 2024-09-16T22:17:02.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026853495000000002, 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18015 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500581+0000", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-07 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500582+0000", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500585+0000", 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9749999999999999e-06 2024-09-16T22:17:02.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500587+0000", 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1679999999999998e-06 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527357+0000", 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026769826 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527398+0000", 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0329e-05 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527435+0000", 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6968999999999998e-05 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18016 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14204928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256817937000001, 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026943981999999998, 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18016 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500590+0000", 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3200000000000001e-07 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500591+0000", 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8199999999999998e-07 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500594+0000", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500597+0000", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527457+0000", 2024-09-16T22:17:02.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026860288 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527497+0000", 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9490999999999998e-05 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527534+0000", 2024-09-16T22:17:02.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.748e-05 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18017 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256808368000002, 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027035519000000001, 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18017 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500600+0000", 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500601+0000", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2200000000000003e-07 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500604+0000", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.959e-06 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500606+0000", 2024-09-16T22:17:02.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527557+0000", 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026950941999999999 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527599+0000", 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1962000000000002e-05 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527635+0000", 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6430000000000002e-05 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18018 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14368768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256799347000001, 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027127043, 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18018 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500609+0000", 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500610+0000", 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500613+0000", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.473e-06 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500616+0000", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2699999999999999e-06 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527659+0000", 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027043754 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527699+0000", 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9847e-05 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527736+0000", 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6610999999999999e-05 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.782 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18019 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14450688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256788778000001, 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027226185, 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18019 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500619+0000", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500620+0000", 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-16T22:17:02.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500623+0000", 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.687e-06 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500627+0000", 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3220000000000001e-06 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500630+0000", 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.146e-06 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527758+0000", 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027128635000000002 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527799+0000", 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1009e-05 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527846+0000", 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6173e-05 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18020 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256775457, 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027315971000000001, 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18020 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500633+0000", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500634+0000", 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:17:02.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500638+0000", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8e-06 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500640+0000", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527869+0000", 2024-09-16T22:17:02.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027229410999999999 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527911+0000", 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1402000000000001e-05 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527949+0000", 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7901000000000002e-05 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18021 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256765426000001, 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027409273000000001, 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18021 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500643+0000", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.23e-07 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500644+0000", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3900000000000003e-07 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500647+0000", 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-16T22:17:02.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500649+0000", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.527972+0000", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027322470000000001 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528013+0000", 2024-09-16T22:17:02.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1211000000000002e-05 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528052+0000", 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9184000000000003e-05 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18022 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14696448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256753374999999, 2024-09-16T22:17:02.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027498129, 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18022 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:17:02.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500655+0000", 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500656+0000", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6099999999999999e-07 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500659+0000", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2150000000000001e-06 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500665+0000", 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:17:02.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528075+0000", 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027410336 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528116+0000", 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1366999999999999e-05 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528153+0000", 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6640999999999997e-05 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18023 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14778368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256740366999999, 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027586167000000002, 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18023 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500668+0000", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500669+0000", 2024-09-16T22:17:02.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7900000000000007e-07 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500673+0000", 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3200000000000001e-06 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500676+0000", 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528176+0000", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027500419000000002 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528217+0000", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0893999999999998e-05 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528254+0000", 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6874e-05 2024-09-16T22:17:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18024 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14860288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256721076000002, 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665443000000001, 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18024 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:17:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500687+0000", 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2e-07 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500689+0000", 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.207e-06 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500692+0000", 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2780000000000002e-06 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500694+0000", 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3939999999999999e-06 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528276+0000", 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027581847 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528317+0000", 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0417999999999998e-05 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528352+0000", 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5879e-05 2024-09-16T22:17:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18025 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256710751, 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02775803, 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18025 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500697+0000", 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500698+0000", 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500699+0000", 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3799999999999996e-07 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500701+0000", 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500707+0000", 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3190000000000002e-06 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528375+0000", 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027668663 2024-09-16T22:17:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528418+0000", 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2073000000000002e-05 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528455+0000", 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7880000000000003e-05 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18026 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15024128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256698246999999, 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027845890000000002, 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18026 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500710+0000", 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3699999999999999e-07 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500711+0000", 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7499999999999998e-07 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500714+0000", 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500717+0000", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528478+0000", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027761082999999999 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528519+0000", 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1090000000000001e-05 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528556+0000", 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7058999999999999e-05 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18027 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15106048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256688906000001, 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027937167999999998, 2024-09-16T22:17:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18027 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500719+0000", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1400000000000001e-07 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500720+0000", 2024-09-16T22:17:02.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500723+0000", 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8439999999999998e-06 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500725+0000", 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528578+0000", 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027852946999999999 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528619+0000", 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1069000000000003e-05 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528656+0000", 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6958000000000001e-05 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18028 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15187968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256679601999998, 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028028412999999999, 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18028 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500729+0000", 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500792+0000", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3039000000000004e-05 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500793+0000", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.734e-06 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500798+0000", 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3579999999999996e-06 2024-09-16T22:17:02.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500800+0000", 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8289999999999999e-06 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528680+0000", 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027879095 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528718+0000", 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8627000000000001e-05 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528757+0000", 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8730999999999999e-05 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18029 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:17:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256602599000001, 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028059634999999999, 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18029 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:17:02.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500806+0000", 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500975+0000", 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016965599999999999 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500977+0000", 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500983+0000", 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2100000000000001e-06 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500986+0000", 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1360000000000001e-06 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528779+0000", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027793629 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528820+0000", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0689999999999998e-05 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528865+0000", 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5145000000000002e-05 2024-09-16T22:17:02.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18030 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256416528999999, 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027977591, 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18030 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.500992+0000", 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501041+0000", 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9799000000000002e-05 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501047+0000", 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0009999999999998e-06 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501052+0000", 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501055+0000", 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528890+0000", 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027835642000000001 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528930+0000", 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0076000000000001e-05 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528969+0000", 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8741000000000001e-05 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:18031 3.17 3:e89c5e27:::rbd_data.11cdc6143ee9.0000000000000001:a [read 15433728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.256348206999998, 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028011306999999999, 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 18031 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.501060+0000", 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505887+0000", 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048273159999999999 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505891+0000", 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.36e-06 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505897+0000", 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7440000000000001e-06 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.505908+0000", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0421000000000001e-05 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.528993+0000", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023084957 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529034+0000", 2024-09-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1023999999999999e-05 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:32.529071+0000", 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7484999999999998e-05 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59481 3.0 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8253457129999999, 2024-09-16T22:17:02.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034053075000000002, 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59481 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932062+0000", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.932064+0000", 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.883e-06 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.936978+0000", 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049139240000000001 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.936996+0000", 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8209000000000001e-05 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937004+0000", 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4220000000000003e-06 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937061+0000", 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7138000000000001e-05 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937157+0000", 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6086000000000001e-05 2024-09-16T22:17:02.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937467+0000", 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000310396 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.949047+0000", 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011579325 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:02.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966042+0000", 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016995415999999999 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966082+0000", 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0046999999999998e-05 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.966115+0000", 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3229000000000003e-05 2024-09-16T22:17:02.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59482 3.0 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head [write 0~0] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8203829900000001, 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030075369000000001, 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:02.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59482 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937025+0000", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937026+0000", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6400000000000002e-07 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937028+0000", 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.979e-06 2024-09-16T22:17:02.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937032+0000", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.937036+0000", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8589999999999998e-06 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943162+0000", 2024-09-16T22:17:02.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061266439999999997 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943248+0000", 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5612999999999995e-05 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.943390+0000", 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142552 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950442+0000", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070514250000000001 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967043+0000", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016601405999999999 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967075+0000", 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1585000000000003e-05 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:02.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.967100+0000", 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582e-05 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:02.856 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:02.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:02.858 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:02.859+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:02.958 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:02.960+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:03.005 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:03.006 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:03.098 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:03.100+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:03.160 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:03.161+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:03.163 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.163 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:03.163 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:03.163 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:03.163 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:03.227 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone27 from 0xbe97fe6 (overlap 0x15f28a) to 0xf5cfa 2024-09-16T22:17:03.264 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:03.262+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:03.324 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:1744 3.1 3:80b62a7f:::rbd_data.11cd70c2eac3.000000000000002c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.732562535, 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014348546, 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1744 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592780+0000", 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592782+0000", 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.22e-06 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592788+0000", 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1120000000000001e-06 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592799+0000", 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592805+0000", 2024-09-16T22:17:03.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592857+0000", 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2256999999999998e-05 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.592904+0000", 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7071999999999999e-05 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.595183+0000", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022788489999999999 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607043+0000", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011859801999999999 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607055+0000", 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759e-05 2024-09-16T22:17:03.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607085+0000", 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0212000000000001e-05 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:22.607129+0000", 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4214000000000001e-05 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.141654987000003, 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:17:03.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:17:03.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.091140297999999, 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:03.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.077814207999999, 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:17:03.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:16275 3.4 3:22d07b81:::rbd_data.11cdc6143ee9.000000000000000c:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.736315695999998, 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025193983, 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 16275 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589027+0000", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589029+0000", 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7630000000000001e-06 2024-09-16T22:17:03.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589037+0000", 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589052+0000", 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4970999999999999e-05 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589058+0000", 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4520000000000002e-06 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589112+0000", 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3953999999999999e-05 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.589229+0000", 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000116301 2024-09-16T22:17:03.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.593369+0000", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004139996 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614089+0000", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020720156 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614108+0000", 2024-09-16T22:17:03.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9216e-05 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614168+0000", 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9913e-05 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:31.614221+0000", 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3254999999999997e-05 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:33503 3.1 3:84128263:::rbd_data.11cd14063e78.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.678047742, 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021447790000000001, 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33503 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647295+0000", 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647296+0000", 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0500000000000002e-07 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647306+0000", 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0061000000000001e-05 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647317+0000", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0502e-05 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647338+0000", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880999999999999e-05 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647393+0000", 2024-09-16T22:17:03.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4376999999999998e-05 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.647469+0000", 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6044000000000006e-05 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.650937+0000", 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003467932 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668661+0000", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017723869 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668674+0000", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3251000000000001e-05 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668707+0000", 2024-09-16T22:17:03.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3599000000000001e-05 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:41.668743+0000", 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5469000000000002e-05 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.022965682999999, 2024-09-16T22:17:03.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:17:03.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:17:03.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:17:03.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42349 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.38148183, 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099116080000000006, 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42349 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943861+0000", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0699999999999999e-07 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943862+0000", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.33e-07 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943865+0000", 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.914e-06 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943867+0000", 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953669+0000", 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098011109999999995 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953726+0000", 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7537999999999997e-05 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953773+0000", 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6567000000000003e-05 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42353 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:17:03.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.381472549, 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010048461, 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42353 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:17:03.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943870+0000", 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943871+0000", 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943872+0000", 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3099999999999996e-07 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943875+0000", 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4450000000000001e-06 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.943883+0000", 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3559999999999993e-06 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953791+0000", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099081020000000002 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953862+0000", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1050999999999994e-05 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.953919+0000", 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6366999999999998e-05 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42585 3.7 e51/23 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head v 51'156, mlcod=51'154)", 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.153949662, 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036096402999999999, 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171395+0000", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174324+0000", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029293370000000002 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174349+0000", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174358+0000", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6979999999999993e-06 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174442+0000", 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3858999999999998e-05 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174470+0000", 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8042999999999999e-05 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207434+0000", 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032963615000000002 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207490+0000", 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831999999999997e-05 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44663 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763472001, 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011080998999999999, 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44663 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561871+0000", 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0499999999999999e-07 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561879+0000", 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5349999999999999e-06 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561886+0000", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2590000000000003e-06 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.561892+0000", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6389999999999997e-06 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570230+0000", 2024-09-16T22:17:03.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.008338715 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.570292+0000", 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1756999999999998e-05 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572952+0000", 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026596889999999998 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44776 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.76325153, 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098382499999999998, 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44776 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562091+0000", 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562092+0000", 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562094+0000", 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562101+0000", 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.024e-06 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562115+0000", 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4054e-05 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571783+0000", 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096675010000000002 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571850+0000", 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.737e-05 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571930+0000", 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9476999999999998e-05 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44777 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 77824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763214894000001, 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099730849999999996, 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44777 2024-09-16T22:17:03.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562128+0000", 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562129+0000", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.51e-07 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562131+0000", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562138+0000", 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562144+0000", 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5049999999999998e-06 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.571953+0000", 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098094210000000005 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572022+0000", 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9190000000000007e-05 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572101+0000", 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8956999999999996e-05 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44778 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763186741, 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010114365, 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44778 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562156+0000", 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562157+0000", 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562158+0000", 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562165+0000", 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5719999999999999e-06 2024-09-16T22:17:03.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562170+0000", 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3499999999999996e-06 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572125+0000", 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099544479999999994 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572193+0000", 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8262000000000005e-05 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572271+0000", 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7411999999999997e-05 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44779 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763164451, 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010260511999999999, 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44779 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562178+0000", 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562179+0000", 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562185+0000", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6740000000000002e-06 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562191+0000", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9410000000000001e-06 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562197+0000", 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:17:03.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572297+0000", 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010099542 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572361+0000", 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4673999999999998e-05 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572439+0000", 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7717000000000001e-05 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44780 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763138735, 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01039753, 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44780 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562204+0000", 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562205+0000", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562207+0000", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.956e-06 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562213+0000", 2024-09-16T22:17:03.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5799999999999997e-06 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562218+0000", 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.028e-06 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572461+0000", 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010243321999999999 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572526+0000", 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4292e-05 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572602+0000", 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5952999999999996e-05 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44781 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763117831000001, 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010560442, 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44781 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562225+0000", 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562228+0000", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562234+0000", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8310000000000003e-06 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562239+0000", 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0080000000000002e-06 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572626+0000", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010386799 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572693+0000", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7008999999999995e-05 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572786+0000", 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2346e-05 2024-09-16T22:17:03.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44782 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763080105, 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010842269, 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44782 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:17:03.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562263+0000", 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9299999999999998e-07 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562266+0000", 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562273+0000", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0670000000000003e-06 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562279+0000", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572810+0000", 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010530684 2024-09-16T22:17:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.572944+0000", 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134387 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573105+0000", 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000161096 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44783 3.4 3:23e6bf5a:::rbd_data.11cd43ed039f.0000000000000010:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.763056091999999, 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011016781999999999, 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44783 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:17:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562287+0000", 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562290+0000", 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2579999999999998e-06 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562296+0000", 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5509999999999996e-06 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.562301+0000", 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573160+0000", 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010858945 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573221+0000", 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0402000000000002e-05 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.573304+0000", 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2807999999999997e-05 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:64425 3.18 e61/23 3:1d9d356b:::rbd_data.11cdb2b6d61b.0000000000000001:head v 61'238, mlcod=59'235)", 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:59.940889+0000", 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3844539739999999, 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010796998, 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.940889+0000", 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.940889+0000", 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.940889+0000", 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4900000000000002e-07 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.943314+0000", 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024250439999999999 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.943338+0000", 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3269e-05 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.943352+0000", 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4608e-05 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.943423+0000", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0461000000000002e-05 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.943445+0000", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2365999999999999e-05 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.951638+0000", 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081929389999999998 2024-09-16T22:17:03.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:59.951686+0000", 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8062000000000002e-05 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.429 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:03.432 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:03.432+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:03.433 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T22:17:03.433 INFO:teuthology.orchestra.run.smithi031.stdout:3138 trunc from 0xbe97fe6 to 0xcf172a7 2024-09-16T22:17:03.433 INFO:teuthology.orchestra.run.smithi031.stdout:3139 write 0xe05212b thru 0xe055ee4 (0x3dba bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3140 trunc from 0xe055ee5 to 0x6694075 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3141 write 0x4a6b242 thru 0x4a6d42f (0x21ee bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3143 read 0x5a85bfc thru 0x5a8c2d6 (0x66db bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3144 write 0x6a2d7ea thru 0x6a38c0c (0xb423 bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3145 read 0x3ebf21a thru 0x3ec953c (0xa323 bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3146 write 0x8ecea7a thru 0x8edbdc3 (0xd34a bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3148 write 0xb5c690c thru 0xb5d5f84 (0xf679 bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3150 read 0x34afc39 thru 0x34bbbb7 (0xbf7f bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3151 write 0x70e817e thru 0x70ed83e (0x56c1 bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3152 read 0x8903940 thru 0x89090db (0x579c bytes) 2024-09-16T22:17:03.434 INFO:teuthology.orchestra.run.smithi031.stdout:3153 writesame 0x4f8ae28 thru 0x4f944d1 data_size 0x5f(0x96aa bytes) 2024-09-16T22:17:03.435 INFO:teuthology.orchestra.run.smithi031.stdout:3155 punch from 0x30a76ab to 0x30a857d, (0xed2 bytes) 2024-09-16T22:17:03.435 INFO:teuthology.orchestra.run.smithi031.stdout:3157 read 0x9118516 thru 0x911bfc9 (0x3ab4 bytes) 2024-09-16T22:17:03.435 INFO:teuthology.orchestra.run.smithi031.stdout:3158 flatten 2024-09-16T22:17:03.435 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:03.435 INFO:teuthology.orchestra.run.smithi031.stdout:3162 writesame 0x345c0dd thru 0x3465df1 data_size 0x1(0x9d15 bytes) 2024-09-16T22:17:03.447 INFO:teuthology.orchestra.run.smithi031.stdout:3163 punch from 0x6d58fb1 to 0x6d59065, (0xb4 bytes) 2024-09-16T22:17:03.447 INFO:teuthology.orchestra.run.smithi031.stdout:3164 write 0x7bdc318 thru 0x7be59c4 (0x96ad bytes) 2024-09-16T22:17:03.449 INFO:teuthology.orchestra.run.smithi031.stdout:3165 read 0x8fd6fb6 thru 0x8fe3cde (0xcd29 bytes) 2024-09-16T22:17:03.450 INFO:teuthology.orchestra.run.smithi031.stdout:3166 write 0x93e6895 thru 0x93f52eb (0xea57 bytes) 2024-09-16T22:17:03.451 INFO:teuthology.orchestra.run.smithi031.stdout:3167 write 0x6672ea9 thru 0x667db7a (0xacd2 bytes) 2024-09-16T22:17:03.459 INFO:teuthology.orchestra.run.smithi031.stdout:3169 write 0xca979d9 thru 0xca9a0d2 (0x26fa bytes) 2024-09-16T22:17:03.462 INFO:teuthology.orchestra.run.smithi031.stdout:3170 write 0x69cb533 thru 0x69cca87 (0x1555 bytes) 2024-09-16T22:17:03.463 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:03.463 INFO:teuthology.orchestra.run.smithi031.stdout:3172 write 0xa5e6fe7 thru 0xa5e86fd (0x1717 bytes) 2024-09-16T22:17:03.465 INFO:teuthology.orchestra.run.smithi031.stdout:3174 trunc from 0xca9a0d3 to 0xd38d5cd 2024-09-16T22:17:03.470 INFO:teuthology.orchestra.run.smithi031.stdout:3175 writesame 0x85aae34 thru 0x85b73ce data_size 0x1(0xc59b bytes) 2024-09-16T22:17:03.492 INFO:teuthology.orchestra.run.smithi031.stdout:3176 punch from 0x2e3d7a9 to 0x2e4c504, (0xed5b bytes) 2024-09-16T22:17:03.506 INFO:teuthology.orchestra.run.smithi031.stdout:3177 trunc from 0xd38d5cd to 0x418d45 2024-09-16T22:17:03.532 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:03.533+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:03.536 INFO:teuthology.orchestra.run.smithi031.stdout:3178 punch from 0x1986cf to 0x19ea40, (0x6371 bytes) 2024-09-16T22:17:03.540 INFO:teuthology.orchestra.run.smithi031.stdout:3179 trunc from 0x418d45 to 0x4000ed5 2024-09-16T22:17:03.559 INFO:teuthology.orchestra.run.smithi031.stdout:3180 write 0xaa93690 thru 0xaa98d0c (0x567d bytes) 2024-09-16T22:17:03.561 INFO:teuthology.orchestra.run.smithi031.stdout:3181 writesame 0x5baee3b thru 0x5bb2c47 data_size 0xf(0x3e0d bytes) 2024-09-16T22:17:03.567 INFO:teuthology.orchestra.run.smithi031.stdout:3182 trunc from 0xaa98d0d to 0xbf5cd32 2024-09-16T22:17:03.572 INFO:teuthology.orchestra.run.smithi031.stdout:3183 read 0x334ff8e thru 0x335a942 (0xa9b5 bytes) 2024-09-16T22:17:03.573 INFO:teuthology.orchestra.run.smithi031.stdout:3184 read 0x64183a1 thru 0x64239f2 (0xb652 bytes) 2024-09-16T22:17:03.573 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:03.573 INFO:teuthology.orchestra.run.smithi031.stdout:3187 read 0x4fd7bd4 thru 0x4fe43bd (0xc7ea bytes) 2024-09-16T22:17:03.573 INFO:teuthology.orchestra.run.smithi031.stdout:3188 punch from 0x1325d8d to 0x1329961, (0x3bd4 bytes) 2024-09-16T22:17:03.574 INFO:teuthology.orchestra.run.smithi031.stdout:3191 punch from 0x37add46 to 0x37b257f, (0x4839 bytes) 2024-09-16T22:17:03.580 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:03.580 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.582 INFO:teuthology.orchestra.run.smithi031.stdout:3192 trunc from 0xbf5cd32 to 0x3a9805f 2024-09-16T22:17:03.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:03.590 INFO:teuthology.orchestra.run.smithi031.stdout:3194 read 0x259abef thru 0x259f5e5 (0x49f7 bytes) 2024-09-16T22:17:03.590 INFO:teuthology.orchestra.run.smithi031.stdout:3195 punch from 0x24bf25 to 0x24dff1, (0x20cc bytes) 2024-09-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi031.stdout:3196 writesame 0xe4c925e thru 0xe4cab9a data_size 0xd(0x193d bytes) 2024-09-16T22:17:03.608 INFO:teuthology.orchestra.run.smithi031.stdout:3197 trunc from 0xe4cab9b to 0xcfebce2 2024-09-16T22:17:03.613 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:03.613 INFO:teuthology.orchestra.run.smithi031.stdout:3199 clone 29 order 23 su 65536 sc 9 2024-09-16T22:17:03.633 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:03.634+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:03.693 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:03.694 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:03.745 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:03.747+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:03.806 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40648 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917662656999999, 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099836779999999993, 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40648 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890093+0000", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890095+0000", 2024-09-16T22:17:03.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890102+0000", 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1679999999999998e-06 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890111+0000", 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.242e-06 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.899922+0000", 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098110300000000001 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.899992+0000", 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0154999999999995e-05 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900077+0000", 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4709000000000005e-05 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40649 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917637724, 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010144003, 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40649 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890118+0000", 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890120+0000", 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0829999999999998e-06 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890127+0000", 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0679999999999999e-06 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890132+0000", 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900103+0000", 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099713340000000001 2024-09-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900174+0000", 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0463000000000006e-05 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900262+0000", 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7903000000000002e-05 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40650 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917616798999999, 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010302911, 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40650 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890139+0000", 2024-09-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8000000000000001e-07 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890141+0000", 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8190000000000001e-06 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890148+0000", 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890156+0000", 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.619e-06 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900295+0000", 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010139005 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900367+0000", 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1810000000000005e-05 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900442+0000", 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4510000000000003e-05 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40651 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917592404000001, 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010449882000000001, 2024-09-16T22:17:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40651 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890163+0000", 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890164+0000", 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9400000000000001e-07 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890166+0000", 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.996e-06 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890173+0000", 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9990000000000002e-06 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890177+0000", 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9099999999999996e-06 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900469+0000", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010291873999999999 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900539+0000", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9448000000000003e-05 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900613+0000", 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4260999999999999e-05 2024-09-16T22:17:03.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40652 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917571646000001, 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01060757, 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40652 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890184+0000", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399999999999998e-07 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890186+0000", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9199999999999998e-06 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890192+0000", 2024-09-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8889999999999999e-06 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890196+0000", 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900642+0000", 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010445217 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900712+0000", 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0009e-05 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900791+0000", 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9779999999999998e-05 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40653 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917550990000001, 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010775066999999999, 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40653 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890205+0000", 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:03.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890212+0000", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.759e-06 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890219+0000", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3379999999999999e-06 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890224+0000", 2024-09-16T22:17:03.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0860000000000001e-06 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900826+0000", 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010601522 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900898+0000", 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1967999999999999e-05 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.900980+0000", 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2025000000000002e-05 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40654 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917524684, 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010923034, 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40654 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890231+0000", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890233+0000", 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890240+0000", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.922e-06 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890245+0000", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8520000000000003e-06 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901007+0000", 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010761517999999999 2024-09-16T22:17:03.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901074+0000", 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901154+0000", 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9721999999999993e-05 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40655 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917504178, 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011076674, 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40655 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890251+0000", 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890252+0000", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-07 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890253+0000", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890259+0000", 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6890000000000001e-06 2024-09-16T22:17:03.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890263+0000", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-06 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901180+0000", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010916762 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901243+0000", 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2841000000000004e-05 2024-09-16T22:17:03.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901328+0000", 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5346000000000006e-05 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40656 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 120832~1115] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917486312999999, 2024-09-16T22:17:03.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011320360999999999, 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40656 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890269+0000", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890270+0000", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890271+0000", 2024-09-16T22:17:03.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7999999999999999e-06 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890278+0000", 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890283+0000", 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7380000000000002e-06 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901365+0000", 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011082277 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901476+0000", 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111156 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901590+0000", 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113634 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:40657 3.b 3:d2111614:::rbd_data.11cd43ed039f.0000000000000019:16 [read 129024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.917466835000001, 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011486682999999999, 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 40657 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890289+0000", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0900000000000002e-07 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890298+0000", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2199999999999998e-06 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890309+0000", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0562e-05 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.890314+0000", 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901629+0000", 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011315218 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901707+0000", 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7736000000000002e-05 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:48.901775+0000", 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8682000000000004e-05 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42587 3.8 e51/23 3:1432c3a7:::rbd_data.11cd49ca855d.0000000000000007:head v 51'187, mlcod=51'185)", 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:03.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.625715925, 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025255219999999998, 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182040+0000", 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.182041+0000", 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8679999999999999e-06 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.184972+0000", 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029307539999999998 2024-09-16T22:17:03.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.184994+0000", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560999999999999e-05 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185003+0000", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3060000000000003e-06 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185066+0000", 2024-09-16T22:17:03.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3235e-05 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.185092+0000", 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207246+0000", 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022153889 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207295+0000", 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8668000000000002e-05 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42734 3.b 3:d3231630:::rbd_data.11cd49ca855d.000000000000000e:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.471078021, 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020378594, 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42734 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336678+0000", 2024-09-16T22:17:03.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336679+0000", 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6270000000000001e-06 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336686+0000", 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4980000000000003e-06 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336698+0000", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2758000000000001e-05 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336705+0000", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6209999999999998e-06 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336766+0000", 2024-09-16T22:17:03.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0659999999999999e-05 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.336771+0000", 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8130000000000003e-06 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.338051+0000", 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001280719 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.338092+0000", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1088999999999999e-05 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.340820+0000", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027281269999999999 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.356965+0000", 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016144807000000001 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.356977+0000", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1474999999999999e-05 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.357009+0000", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2561999999999998e-05 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.357056+0000", 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6838000000000001e-05 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44290 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257838716, 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099311009999999995, 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44290 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549917+0000", 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-07 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549919+0000", 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5400000000000001e-06 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549924+0000", 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5849999999999999e-06 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549929+0000", 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5560000000000001e-06 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559836+0000", 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009907585 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559848+0000", 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1522000000000001e-05 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44291 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257821566000001, 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099459179999999998, 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44291 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549934+0000", 2024-09-16T22:17:03.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.15e-07 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549936+0000", 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-06 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549941+0000", 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.614e-06 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549946+0000", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4499999999999997e-06 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559871+0000", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099246149999999995 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559880+0000", 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3989999999999993e-06 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44292 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257804481000001, 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099693360000000005, 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44292 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549951+0000", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3799999999999998e-07 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549953+0000", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.502e-06 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549959+0000", 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549963+0000", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7389999999999999e-06 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559912+0000", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099484870000000006 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559920+0000", 2024-09-16T22:17:03.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5299999999999996e-06 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44293 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257772767000001, 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099827490000000008, 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44293 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549983+0000", 2024-09-16T22:17:03.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549984+0000", 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6600000000000005e-07 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549988+0000", 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0210000000000001e-06 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.549996+0000", 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4479999999999994e-06 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550003+0000", 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6309999999999997e-06 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559954+0000", 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099515709999999993 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.559966+0000", 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1212e-05 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44294 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257740933000001, 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099995169999999994, 2024-09-16T22:17:03.885 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44294 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550015+0000", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550018+0000", 2024-09-16T22:17:03.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4959999999999999e-06 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550025+0000", 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3139999999999998e-06 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550030+0000", 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5589999999999996e-06 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560002+0000", 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099713679999999996 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560014+0000", 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2296000000000001e-05 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.889 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:44295 3.b 3.2868884b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.257717728999999, 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010122055, 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44295 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550038+0000", 2024-09-16T22:17:03.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550048+0000", 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8579999999999995e-06 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550056+0000", 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4309999999999997e-06 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.550061+0000", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7679999999999997e-06 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560151+0000", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010089677999999999 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:50.560160+0000", 2024-09-16T22:17:03.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8519999999999993e-06 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:59483 3.c e60/23 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head v 60'175, mlcod=59'173)", 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8682345189999996, 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018712162000000001, 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.939521+0000", 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4900000000000002e-07 2024-09-16T22:17:03.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946524+0000", 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070025319999999997 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946548+0000", 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4498000000000002e-05 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946560+0000", 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1539000000000001e-05 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946637+0000", 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6970000000000003e-05 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.946662+0000", 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5369000000000001e-05 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958202+0000", 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011539667999999999 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958233+0000", 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1337000000000002e-05 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:59484 3.c e60/23 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head v 60'177, mlcod=59'173)", 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8600038410000002, 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010812252, 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947752+0000", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6000000000000003e-07 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947758+0000", 2024-09-16T22:17:03.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999998e-06 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947768+0000", 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0094e-05 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.947774+0000", 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3130000000000004e-06 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950861+0000", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030862279999999999 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.950876+0000", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5722e-05 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958535+0000", 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076586629999999996 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:03.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.958564+0000", 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9051999999999999e-05 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:03.903 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:03.905 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:03.906+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:03.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:04.005 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:04.006+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:04.017 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.017 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:04.017 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:04.018 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:04.106 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:04.108+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:04.134 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.134 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:04.134 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:04.134 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:04.134 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:04.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:17:04.229 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:04.231+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42086 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 161792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980145+0000", 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284238242000001, 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021077230999999998, 2024-09-16T22:17:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42086 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980145+0000", 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980145+0000", 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980146+0000", 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980148+0000", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.922e-06 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980154+0000", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980159+0000", 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8890000000000001e-06 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001092+0000", 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020933384999999999 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001153+0000", 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0915000000000002e-05 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001223+0000", 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9293000000000002e-05 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42087 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980165+0000", 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284218407000001, 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021220460999999999, 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42087 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980165+0000", 2024-09-16T22:17:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980165+0000", 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980166+0000", 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980173+0000", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7940000000000003e-06 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980180+0000", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.409e-06 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980185+0000", 2024-09-16T22:17:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.668e-06 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001251+0000", 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021066788999999999 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001314+0000", 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2597999999999999e-05 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001386+0000", 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1780999999999996e-05 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42088 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 178176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284192084000001, 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021352987, 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42088 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9699999999999996e-07 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980194+0000", 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980200+0000", 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2580000000000001e-06 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980205+0000", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6399999999999996e-06 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001414+0000", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021208751000000001 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001482+0000", 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7868999999999997e-05 2024-09-16T22:17:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001545+0000", 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2821e-05 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42089 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 186368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284092193999999, 2024-09-16T22:17:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021459101000000001, 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42089 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980292+0000", 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980295+0000", 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980303+0000", 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8490000000000006e-06 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980309+0000", 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.947e-06 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001584+0000", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021275051999999999 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001666+0000", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1706999999999995e-05 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001751+0000", 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4906000000000003e-05 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42090 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 194560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284066064999999, 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021544528, 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42090 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980320+0000", 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980327+0000", 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5649999999999998e-06 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980332+0000", 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001788+0000", 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021456295 2024-09-16T22:17:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001826+0000", 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8083000000000001e-05 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001862+0000", 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5722999999999997e-05 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42091 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284044859, 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021714943, 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42091 2024-09-16T22:17:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980341+0000", 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9180000000000001e-06 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980348+0000", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980353+0000", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9339999999999997e-06 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001898+0000", 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021545557 2024-09-16T22:17:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001979+0000", 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0061999999999993e-05 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002054+0000", 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5160000000000005e-05 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42092 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 210944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.284023887, 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021811192, 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42092 2024-09-16T22:17:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980369+0000", 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6959999999999994e-06 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980376+0000", 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8789999999999997e-06 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980388+0000", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1996e-05 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002082+0000", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021694083999999999 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002131+0000", 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000003e-05 2024-09-16T22:17:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002171+0000", 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9836000000000003e-05 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42093 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 219136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283456958, 2024-09-16T22:17:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021327148000000001, 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42093 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-07 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980931+0000", 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4960000000000001e-06 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980942+0000", 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0757999999999999e-05 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980950+0000", 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-06 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002187+0000", 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021237018999999999 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002220+0000", 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2861000000000003e-05 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002254+0000", 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3818e-05 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42094 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 227328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283422805000001, 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021427170999999998, 2024-09-16T22:17:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42094 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980964+0000", 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4150000000000002e-06 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980970+0000", 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6429999999999998e-06 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980984+0000", 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3259e-05 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002268+0000", 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021284416 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002325+0000", 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002388+0000", 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42095 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 235520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283392066999999, 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021555900999999999, 2024-09-16T22:17:04.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42095 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:04.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980994+0000", 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981001+0000", 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981007+0000", 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8189999999999997e-06 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002413+0000", 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021406027000000001 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002477+0000", 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4243000000000001e-05 2024-09-16T22:17:04.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002547+0000", 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9980000000000004e-05 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42096 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 243712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283368876000001, 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021688049000000001, 2024-09-16T22:17:04.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42096 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981017+0000", 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981023+0000", 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981028+0000", 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1370000000000001e-06 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002575+0000", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021546684999999999 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002637+0000", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1950000000000001e-05 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002703+0000", 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6268999999999999e-05 2024-09-16T22:17:04.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42097 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 251904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283349080000001, 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021820512, 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42097 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981037+0000", 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7889999999999999e-06 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981044+0000", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.396e-06 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981049+0000", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2669999999999997e-06 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002728+0000", 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021678669000000001 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002787+0000", 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8749000000000003e-05 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002855+0000", 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8162000000000002e-05 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42098 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 260096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.283326347999999, 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021928260000000002, 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42098 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981058+0000", 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981066+0000", 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1710000000000002e-06 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981074+0000", 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.746e-06 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981079+0000", 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002883+0000", 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021803978000000002 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002941+0000", 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8700999999999999e-05 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002986+0000", 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4073000000000003e-05 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42763 3.15 3:ae84599f:::rbd_data.11cd49ca855d.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-16T22:17:04.327 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.882143836000001, 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024874701999999999, 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42763 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382241+0000", 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5899999999999995e-07 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382245+0000", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382255+0000", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9299999999999998e-06 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382260+0000", 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382313+0000", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3091999999999999e-05 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382359+0000", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6866999999999997e-05 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.386616+0000", 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042566100000000001 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407035+0000", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020419085 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407047+0000", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1836e-05 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407080+0000", 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2892999999999998e-05 2024-09-16T22:17:04.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407114+0000", 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4569000000000001e-05 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59483 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3317754559999999, 2024-09-16T22:17:04.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028869210999999999, 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59483 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932611+0000", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8710000000000001e-06 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936104+0000", 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034931049999999998 2024-09-16T22:17:04.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936128+0000", 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3751e-05 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936140+0000", 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2428e-05 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936233+0000", 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2406999999999998e-05 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936414+0000", 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018144400000000001 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.937091+0000", 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067658099999999999 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958379+0000", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02128855 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961342+0000", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002962667 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961400+0000", 2024-09-16T22:17:04.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8335000000000003e-05 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961477+0000", 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7071999999999997e-05 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59484 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [write 0~0] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3251948709999999, 2024-09-16T22:17:04.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025486099000000002, 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59484 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:04.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939190+0000", 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.716e-06 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939198+0000", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4800000000000004e-06 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939219+0000", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0851000000000001e-05 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939229+0000", 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0599e-05 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947061+0000", 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078311649999999993 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947144+0000", 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3531999999999999e-05 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:04.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947346+0000", 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020164299999999999 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958658+0000", 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011312467 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964560+0000", 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059020119999999999 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964626+0000", 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6025999999999994e-05 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964675+0000", 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8607999999999999e-05 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59481 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'145, mlcod=59'143)", 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3242978140000004, 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025627553000000001, 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940088+0000", 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1660000000000001e-06 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945430+0000", 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053420330000000004 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945444+0000", 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3787e-05 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945453+0000", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945526+0000", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3166999999999999e-05 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945549+0000", 2024-09-16T22:17:04.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3666e-05 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965653+0000", 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020104052000000001 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965713+0000", 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9960999999999997e-05 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59485 3.1d e60/23 3:bcb688ae:::rbd_data.11cdc742ac9f.0000000000000002:head v 60'169, mlcod=59'167)", 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3235160600000002, 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025808811000000001, 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8400000000000003e-07 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944623+0000", 2024-09-16T22:17:04.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003754949 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944642+0000", 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8539999999999999e-05 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944654+0000", 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1905e-05 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944767+0000", 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113485 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944793+0000", 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5459000000000001e-05 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966625+0000", 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021832767999999999 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966676+0000", 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0921000000000003e-05 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59482 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'147, mlcod=59'143)", 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3188926070000004, 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021422455999999999, 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945492+0000", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.64e-07 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945500+0000", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6260000000000003e-06 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945506+0000", 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4910000000000002e-06 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945511+0000", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951314+0000", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058029509999999998 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951341+0000", 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7387000000000001e-05 2024-09-16T22:17:04.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966873+0000", 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015531978 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966913+0000", 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:64678 3.f 3:f2f24fe2:::rbd_data.11cdb2b6d61b.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.9796574429999998, 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025102092999999999, 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 64678 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284728+0000", 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.381e-06 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284733+0000", 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5509999999999999e-06 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284744+0000", 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294999999999999e-05 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284750+0000", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284795+0000", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5697e-05 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284893+0000", 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7775000000000005e-05 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.288943+0000", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040499730000000001 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309744+0000", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020800511000000001 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309757+0000", 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3165e-05 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309793+0000", 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6183999999999998e-05 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309828+0000", 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5326e-05 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.366 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:04.367 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T22:17:04.368 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:04.368+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:04.368 INFO:teuthology.orchestra.run.smithi031.stdout:3202 punch from 0x708a8f5 to 0x708b75f, (0xe6a bytes) 2024-09-16T22:17:04.372 INFO:teuthology.orchestra.run.smithi031.stdout:3203 writesame 0x75c0f9d thru 0x75c4bc4 data_size 0x1(0x3c28 bytes) 2024-09-16T22:17:04.386 INFO:teuthology.orchestra.run.smithi031.stdout:3204 writesame 0xabae6a6 thru 0xabb177d data_size 0xc(0x30d8 bytes) 2024-09-16T22:17:04.394 INFO:teuthology.orchestra.run.smithi031.stdout:3205 write 0x438442f thru 0x4389049 (0x4c1b bytes) 2024-09-16T22:17:04.402 INFO:teuthology.orchestra.run.smithi031.stdout:3206 read 0xb83a18d thru 0xb83fac0 (0x5934 bytes) 2024-09-16T22:17:04.414 INFO:teuthology.orchestra.run.smithi031.stdout:3207 trunc from 0xcfebce2 to 0x2db2859 2024-09-16T22:17:04.461 INFO:teuthology.orchestra.run.smithi031.stdout:3208 read 0xdcf349 thru 0xdd1b3e (0x27f6 bytes) 2024-09-16T22:17:04.467 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:04.469+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:04.473 INFO:teuthology.orchestra.run.smithi031.stdout:3209 trunc from 0x2db2859 to 0xc489442 2024-09-16T22:17:04.479 INFO:teuthology.orchestra.run.smithi031.stdout:3210 write 0x43d6e41 thru 0x43e2587 (0xb747 bytes) 2024-09-16T22:17:04.480 INFO:teuthology.orchestra.run.smithi031.stdout:3211 write 0x4c80148 thru 0x4c8077a (0x633 bytes) 2024-09-16T22:17:04.483 INFO:teuthology.orchestra.run.smithi031.stdout:3212 writesame 0x132528 thru 0x13f903 data_size 0x5b(0xd3dc bytes) 2024-09-16T22:17:04.486 INFO:teuthology.orchestra.run.smithi031.stdout:3215 punch from 0x7b4aafe to 0x7b5265c, (0x7b5e bytes) 2024-09-16T22:17:04.486 INFO:teuthology.orchestra.run.smithi031.stdout:3217 write 0x62a52e thru 0x62c479 (0x1f4c bytes) 2024-09-16T22:17:04.500 INFO:teuthology.orchestra.run.smithi031.stdout:3218 trunc from 0xc489442 to 0x162552d 2024-09-16T22:17:04.500 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.501 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:04.501 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:04.501 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:04.512 INFO:teuthology.orchestra.run.smithi031.stdout:3220 read 0x159a362 thru 0x159fdd4 (0x5a73 bytes) 2024-09-16T22:17:04.513 INFO:teuthology.orchestra.run.smithi031.stdout:3222 read 0xd1d8b3 thru 0xd2b371 (0xdabf bytes) 2024-09-16T22:17:04.514 INFO:teuthology.orchestra.run.smithi031.stdout:3224 trunc from 0x162552d to 0x235095c 2024-09-16T22:17:04.519 INFO:teuthology.orchestra.run.smithi031.stdout:3226 read 0x9e3587 thru 0x9f00d1 (0xcb4b bytes) 2024-09-16T22:17:04.525 INFO:teuthology.orchestra.run.smithi031.stdout:3227 write 0x2b5ae5e thru 0x2b5d6ea (0x288d bytes) 2024-09-16T22:17:04.549 INFO:teuthology.orchestra.run.smithi031.stdout:3228 write 0xdde1091 thru 0xdde1dc9 (0xd39 bytes) 2024-09-16T22:17:04.552 INFO:teuthology.orchestra.run.smithi031.stdout:3229 read 0x597d5e8 thru 0x597e681 (0x109a bytes) 2024-09-16T22:17:04.552 INFO:teuthology.orchestra.run.smithi031.stdout:3230 read 0xc63bdc thru 0xc6f752 (0xbb77 bytes) 2024-09-16T22:17:04.553 INFO:teuthology.orchestra.run.smithi031.stdout:3231 write 0x3728315 thru 0x3730ca7 (0x8993 bytes) 2024-09-16T22:17:04.554 INFO:teuthology.orchestra.run.smithi031.stdout:3233 punch from 0xa174dc0 to 0xa17e9d5, (0x9c15 bytes) 2024-09-16T22:17:04.554 INFO:teuthology.orchestra.run.smithi031.stdout:3234 write 0x9228f8f thru 0x9234315 (0xb387 bytes) 2024-09-16T22:17:04.558 INFO:teuthology.orchestra.run.smithi031.stdout:3235 read 0x39199f1 thru 0x3923c81 (0xa291 bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3236 punch from 0x912931c to 0x912c001, (0x2ce5 bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3237 read 0x379b515 thru 0x379ed15 (0x3801 bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3238 punch from 0xc1cd767 to 0xc1dd1b1, (0xfa4a bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3239 punch from 0xa026622 to 0xa02cb87, (0x6565 bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3240 punch from 0x8a8dd0f to 0x8a9439b, (0x668c bytes) 2024-09-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:3241 write 0x2e057c9 thru 0x2e0eb56 (0x938e bytes) 2024-09-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi031.stdout:3243 read 0x7496a45 thru 0x74992ab (0x2867 bytes) 2024-09-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi031.stdout:3244 clone 30 order 19 su 4096 sc 5 2024-09-16T22:17:04.568 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:04.570+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:04.626 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:04.626 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:04.626 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:04.626 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:04.669 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:04.671+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:04.769 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:04.771+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55222 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4317184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4491053010000003, 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063938485000000003, 2024-09-16T22:17:04.785 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55222 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336759+0000", 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000208063 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336764+0000", 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336776+0000", 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.182e-05 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336782+0000", 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7789999999999998e-06 2024-09-16T22:17:04.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400086+0000", 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063303645000000006 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400310+0000", 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022399299999999999 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400489+0000", 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017931300000000001 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55223 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4488619680000001, 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911932000000005, 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55223 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336950+0000", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155821 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336954+0000", 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:17:04.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336965+0000", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1055e-05 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336972+0000", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4309999999999999e-06 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400531+0000", 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559177999999994 2024-09-16T22:17:04.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400612+0000", 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1360999999999996e-05 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400706+0000", 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3837000000000003e-05 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55224 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4333568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4486658610000003, 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064193369, 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55224 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337203+0000", 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021318899999999999 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337208+0000", 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1710000000000004e-06 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337215+0000", 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3089999999999998e-06 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337219+0000", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400749+0000", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063530160000000002 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401000+0000", 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000250995 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401184+0000", 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018346399999999999 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55225 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4919296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4484290499999997, 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064167392000000004, 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55225 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337451+0000", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022388599999999999 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337456+0000", 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7280000000000003e-06 2024-09-16T22:17:04.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337466+0000", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0351e-05 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337473+0000", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7449999999999998e-06 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401218+0000", 2024-09-16T22:17:04.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063745183999999996 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401298+0000", 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0302999999999993e-05 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401394+0000", 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6194999999999996e-05 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55226 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4349952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4481718939999997, 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064121972999999999, 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55226 2024-09-16T22:17:04.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337709+0000", 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022441200000000001 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337713+0000", 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337724+0000", 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0740999999999999e-05 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337731+0000", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401426+0000", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063695389000000005 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401507+0000", 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0356000000000001e-05 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401606+0000", 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9631999999999997e-05 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55227 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4935680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4479131540000001, 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064076279, 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55227 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337889+0000", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146055 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337894+0000", 2024-09-16T22:17:04.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337901+0000", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337905+0000", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1080000000000003e-06 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401637+0000", 2024-09-16T22:17:04.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063731333000000001 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401721+0000", 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3969999999999997e-05 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401819+0000", 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55228 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4366336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4477427020000002, 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064169421000000004, 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55228 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338139+0000", 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022557499999999999 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338143+0000", 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338158+0000", 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4498000000000001e-05 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338165+0000", 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401880+0000", 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063715331 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401978+0000", 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8158000000000004e-05 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402083+0000", 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104799 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55229 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4474037769999999, 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064053115999999993, 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55229 2024-09-16T22:17:04.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338426+0000", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017367500000000001 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338431+0000", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338442+0000", 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1749999999999999e-05 2024-09-16T22:17:04.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338449+0000", 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6649999999999997e-06 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402117+0000", 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063668279999999994 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402207+0000", 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9257000000000003e-05 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402305+0000", 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8911000000000002e-05 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55231 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4968448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4468255870000002, 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911281, 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55231 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-07 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338834+0000", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338841+0000", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338847+0000", 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1360000000000002e-06 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402566+0000", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063718876999999993 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402647+0000", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1384000000000005e-05 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402742+0000", 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4686999999999997e-05 2024-09-16T22:17:04.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55232 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4399104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4468005850000001, 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064122381000000006, 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55232 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339005+0000", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014984300000000001 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339010+0000", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999999e-06 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339023+0000", 2024-09-16T22:17:04.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2904e-05 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339030+0000", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402777+0000", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063746612999999994 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402864+0000", 2024-09-16T22:17:04.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7818000000000005e-05 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402978+0000", 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113507 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55233 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4984832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4466146389999999, 2024-09-16T22:17:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151505999999997, 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55233 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339210+0000", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016822699999999999 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:04.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.741e-06 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339225+0000", 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0149000000000001e-05 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339232+0000", 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403011+0000", 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063779340000000004 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403096+0000", 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5482000000000006e-05 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403193+0000", 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6496999999999994e-05 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55234 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4464082759999997, 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151969000000003, 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55234 2024-09-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339412+0000", 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000164544 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339417+0000", 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6160000000000004e-06 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339428+0000", 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0522e-05 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339435+0000", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7880000000000007e-06 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403227+0000", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063791688999999999 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403309+0000", 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2254000000000002e-05 2024-09-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403400+0000", 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0556000000000005e-05 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55235 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5001216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4462086830000001, 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064168931999999998, 2024-09-16T22:17:04.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55235 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339684+0000", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023680199999999999 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339690+0000", 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.049e-06 2024-09-16T22:17:04.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339701+0000", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0494e-05 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339706+0000", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6579999999999999e-06 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403431+0000", 2024-09-16T22:17:04.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063724905999999998 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403519+0000", 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7892999999999993e-05 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403616+0000", 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.713e-05 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55236 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4431872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4459390269999997, 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064115839999999993, 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55236 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339965+0000", 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024771199999999998 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339970+0000", 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339980+0000", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0254999999999999e-05 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339987+0000", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7519999999999999e-06 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403650+0000", 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063663553999999997 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403736+0000", 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5778000000000004e-05 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403833+0000", 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6854000000000005e-05 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55237 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5017600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4456570659999999, 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064058452000000002, 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55237 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340000+0000", 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340002+0000", 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5859999999999999e-06 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340009+0000", 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2080000000000003e-06 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340015+0000", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403865+0000", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063850812000000007 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403955+0000", 2024-09-16T22:17:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9648999999999995e-05 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404057+0000", 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102472 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55238 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4448256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4456324880000002, 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064252408999999996, 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55238 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340290+0000", 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026611899999999999 2024-09-16T22:17:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340294+0000", 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2810000000000002e-06 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340302+0000", 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4630000000000001e-06 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340308+0000", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8259999999999998e-06 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404092+0000", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063783230999999996 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404178+0000", 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6080000000000003e-05 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404276+0000", 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8408999999999999e-05 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55239 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5033984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4453359370000003, 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064172565000000001, 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55239 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341020+0000", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070000599999999998 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341025+0000", 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6589999999999998e-06 2024-09-16T22:17:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341038+0000", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3534e-05 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341046+0000", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6179999999999997e-06 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404308+0000", 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063261957999999993 2024-09-16T22:17:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404393+0000", 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4791999999999997e-05 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404493+0000", 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9998e-05 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55243 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5066752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4444724730000003, 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064032600999999995, 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55243 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341467+0000", 2024-09-16T22:17:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028289899999999998 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341471+0000", 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5569999999999997e-06 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341481+0000", 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0339e-05 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341488+0000", 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405047+0000", 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559063999999998 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405129+0000", 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1919e-05 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405216+0000", 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7230000000000003e-05 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55244 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4497408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4441567859999997, 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063903336000000005, 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55244 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341673+0000", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017377700000000001 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341678+0000", 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341688+0000", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0200000000000001e-05 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341695+0000", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.55e-06 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405249+0000", 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063554352999999994 2024-09-16T22:17:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405323+0000", 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3677000000000006e-05 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405403+0000", 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9628000000000004e-05 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55245 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5083136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4439476820000001, 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063879072999999995, 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55245 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341930+0000", 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022177199999999999 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341940+0000", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341950+0000", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0124e-05 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341959+0000", 2024-09-16T22:17:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7260000000000008e-06 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405436+0000", 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063477512 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405509+0000", 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2156999999999994e-05 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405587+0000", 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8957999999999999e-05 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:04.877 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:04.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:04.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:04.880 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:04.880+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:04.979 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:04.981+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:05.030 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:05.030 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:05.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:05.045 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:05.063 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:05.063 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:05.080 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:05.081 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:05.082+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:05.090 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:05.090 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:05.091 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T22:17:05.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T22:17:05.096 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:05.115 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:05.115 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:05.133 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:05.133 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:05.147 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.147 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.150 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:05.150 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:05.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:05.168 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:05.168 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:05.171 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:17:05.182 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:05.183+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:05.239 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone29 from 0xdde1dca (overlap 0x162552d) to 0x13ee642 2024-09-16T22:17:05.282 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:05.284+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:05.294 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.294 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T22:17:05.384 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:05.385+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:05.425 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.425 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 66, 2024-09-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "57fe7da5-e356-48a4-ba33-5558b1026a32", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-16T22:15:44.319460+0000", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-16T22:17:05.167758+0000", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-16T22:15:56.374983+0000", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-16T22:15:46.438697+0000", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:15:56.583719+0000", 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:17:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:17:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-16T22:17:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:02.234097+0000", 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:17:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-16T22:17:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "22", 2024-09-16T22:17:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 22, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:17:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:17:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-16T22:16:13.789572+0000", 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T22:17:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-16T22:17:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "66", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 32, 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 66, 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-16T22:17:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-16T22:17:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-16T22:17:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-16T22:17:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "44de8de0-e16a-4325-94b9-1cc61f4829db", 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-16T22:17:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2585124124 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2585124124", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2585124124", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2585124124", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2585124124", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0e05fbd1-9707-40b8-91a3-198c6f5b7865", 2024-09-16T22:17:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-16T22:17:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-16T22:17:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3271285223 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/3271285223", 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/3271285223", 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/3271285223", 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/3271285223", 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "b080bc29-0a92-4200-ae3f-0a17a1722145", 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874892035 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2874892035", 2024-09-16T22:17:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2874892035", 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2874892035", 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2874892035", 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "0bc64c75-486c-40ac-abaf-381e151623bd", 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-16T22:17:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-16T22:17:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-16T22:17:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3768411773 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/3768411773", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/3768411773", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/3768411773", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/3768411773", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fae627d1-6bc2-4e5b-85e1-334e5e05f830", 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6824", 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6825", 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6826", 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6827", 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6830", 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6831", 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6828", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6829", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3421901083 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6825/3421901083", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6827/3421901083", 2024-09-16T22:17:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6831/3421901083", 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6829/3421901083", 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "989a79cd-ef84-4bf3-a64f-9d101d0b29d7", 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 58, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6816", 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6817", 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6818", 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6819", 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6822", 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6823", 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6820", 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6821", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 335833224 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6817/335833224", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6819/335833224", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6823/335833224", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6821/335833224", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "92f145b7-e868-43fd-9abe-68c43a407ae2", 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 48, 2024-09-16T22:17:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6808", 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6809", 2024-09-16T22:17:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6810", 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6811", 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6814", 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6815", 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6812", 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6813", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1592419018 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6809/1592419018", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6811/1592419018", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6815/1592419018", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6813/1592419018", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "373f8002-53a4-4ca7-bbbf-70e8d56b3571", 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 23, 2024-09-16T22:17:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6800", 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6801", 2024-09-16T22:17:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6802", 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6803", 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6806", 2024-09-16T22:17:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6807", 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T22:17:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6804", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.92:6805", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3448871901 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.92:6801/3448871901", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.92:6803/3448871901", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.92:6807/3448871901", 2024-09-16T22:17:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.92:6805/3448871901", 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.033837+0000", 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.799099+0000", 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.769970+0000", 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.965029+0000", 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.706036+0000", 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:53.158188+0000", 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.722931+0000", 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-16T22:17:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540719139924082687, 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-16T22:15:52.485009+0000", 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.0", 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: 0 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-16T22:17:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:05.512 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:05.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:05.515 INFO:tasks.thrashosds.thrasher:j is {'epoch': 66, 'fsid': '57fe7da5-e356-48a4-ba33-5558b1026a32', 'created': '2024-09-16T22:15:44.319460+0000', 'modified': '2024-09-16T22:17:05.167758+0000', 'last_up_change': '2024-09-16T22:15:56.374983+0000', 'last_in_change': '2024-09-16T22:15:46.438697+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T22:15:56.583719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T22:16:02.234097+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T22:16:13.789572+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '44de8de0-e16a-4325-94b9-1cc61f4829db', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2585124124}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2585124124}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2585124124}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2585124124}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2585124124}]}, 'public_addr': '172.21.15.31:6817/2585124124', 'cluster_addr': '172.21.15.31:6819/2585124124', 'heartbeat_back_addr': '172.21.15.31:6823/2585124124', 'heartbeat_front_addr': '172.21.15.31:6821/2585124124', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0e05fbd1-9707-40b8-91a3-198c6f5b7865', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 3271285223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 3271285223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 3271285223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 3271285223}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 3271285223}]}, 'public_addr': '172.21.15.31:6801/3271285223', 'cluster_addr': '172.21.15.31:6803/3271285223', 'heartbeat_back_addr': '172.21.15.31:6807/3271285223', 'heartbeat_front_addr': '172.21.15.31:6805/3271285223', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'b080bc29-0a92-4200-ae3f-0a17a1722145', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2874892035}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2874892035}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2874892035}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2874892035}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2874892035}]}, 'public_addr': '172.21.15.31:6809/2874892035', 'cluster_addr': '172.21.15.31:6811/2874892035', 'heartbeat_back_addr': '172.21.15.31:6815/2874892035', 'heartbeat_front_addr': '172.21.15.31:6813/2874892035', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0bc64c75-486c-40ac-abaf-381e151623bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 3768411773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 3768411773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 3768411773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3768411773}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 3768411773}]}, 'public_addr': '172.21.15.31:6825/3768411773', 'cluster_addr': '172.21.15.31:6827/3768411773', 'heartbeat_back_addr': '172.21.15.31:6831/3768411773', 'heartbeat_front_addr': '172.21.15.31:6829/3768411773', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fae627d1-6bc2-4e5b-85e1-334e5e05f830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 3421901083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3421901083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3421901083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3421901083}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3421901083}]}, 'public_addr': '172.21.15.92:6825/3421901083', 'cluster_addr': '172.21.15.92:6827/3421901083', 'heartbeat_back_addr': '172.21.15.92:6831/3421901083', 'heartbeat_front_addr': '172.21.15.92:6829/3421901083', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '989a79cd-ef84-4bf3-a64f-9d101d0b29d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 335833224}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 335833224}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 335833224}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 335833224}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 335833224}]}, 'public_addr': '172.21.15.92:6817/335833224', 'cluster_addr': '172.21.15.92:6819/335833224', 'heartbeat_back_addr': '172.21.15.92:6823/335833224', 'heartbeat_front_addr': '172.21.15.92:6821/335833224', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '92f145b7-e868-43fd-9abe-68c43a407ae2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1592419018}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 1592419018}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 1592419018}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 1592419018}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 1592419018}]}, 'public_addr': '172.21.15.92:6809/1592419018', 'cluster_addr': '172.21.15.92:6811/1592419018', 'heartbeat_back_addr': '172.21.15.92:6815/1592419018', 'heartbeat_front_addr': '172.21.15.92:6813/1592419018', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '373f8002-53a4-4ca7-bbbf-70e8d56b3571', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 3448871901}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 3448871901}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 3448871901}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 3448871901}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 3448871901}]}, 'public_addr': '172.21.15.92:6801/3448871901', 'cluster_addr': '172.21.15.92:6803/3448871901', 'heartbeat_back_addr': '172.21.15.92:6807/3448871901', 'heartbeat_front_addr': '172.21.15.92:6805/3448871901', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.033837+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.799099+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.769970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.965029+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.706036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:53.158188+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.722931+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T22:15:52.485009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.0', 'osds': [5, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T22:17:05.516 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.0 2024-09-16T22:17:05.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.0 2024-09-16T22:17:05.516 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:05.516+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:05.543 INFO:teuthology.orchestra.run.smithi031.stdout:3245 read 0xafdec04 thru 0xafee81e (0xfc1b bytes) 2024-09-16T22:17:05.543 INFO:teuthology.orchestra.run.smithi031.stdout:3250 punch from 0x3dd1267 to 0x3dd3421, (0x21ba bytes) 2024-09-16T22:17:05.556 INFO:teuthology.orchestra.run.smithi031.stdout:3251 trunc from 0xdde1dca to 0x63e910a 2024-09-16T22:17:05.588 INFO:teuthology.orchestra.run.smithi031.stdout:3252 write 0xd975bb2 thru 0xd979db9 (0x4208 bytes) 2024-09-16T22:17:05.599 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.599 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.599 INFO:teuthology.orchestra.run.smithi031.stdout:3256 writesame 0x640bcb2 thru 0x6416d5a data_size 0x87(0xb0a9 bytes) 2024-09-16T22:17:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:3257 punch from 0xbe581ab to 0xbe6701f, (0xee74 bytes) 2024-09-16T22:17:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:3258 write 0xae2d7ca thru 0xae3c02e (0xe865 bytes) 2024-09-16T22:17:05.614 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:05.616+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:05.624 INFO:teuthology.orchestra.run.smithi031.stdout:3259 read 0xd1a56e6 thru 0xd1b3bff (0xe51a bytes) 2024-09-16T22:17:05.624 INFO:teuthology.orchestra.run.smithi031.stdout:3260 write 0x13bb940 thru 0x13c2cd8 (0x7399 bytes) 2024-09-16T22:17:05.657 INFO:teuthology.orchestra.run.smithi031.stdout:3261 write 0x6f69fa7 thru 0x6f75d70 (0xbdca bytes) 2024-09-16T22:17:05.665 INFO:teuthology.orchestra.run.smithi031.stdout:3262 write 0xf82f63 thru 0xf8d934 (0xa9d2 bytes) 2024-09-16T22:17:05.689 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.689 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.699 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:05.701 INFO:teuthology.orchestra.run.smithi031.stdout:3264 write 0xd10b1cb thru 0xd118c59 (0xda8f bytes) 2024-09-16T22:17:05.709 INFO:teuthology.orchestra.run.smithi031.stdout:3265 trunc from 0xd979dba to 0xa7dbfee 2024-09-16T22:17:05.731 INFO:teuthology.orchestra.run.smithi031.stdout:3266 write 0xe496077 thru 0xe4966ba (0x644 bytes) 2024-09-16T22:17:05.731 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:05.733+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:05.734 INFO:teuthology.orchestra.run.smithi031.stdout:3267 read 0xbaffee thru 0xbbb1b5 (0xb1c8 bytes) 2024-09-16T22:17:05.736 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.736 INFO:teuthology.orchestra.run.smithi031.stdout:3270 write 0x5d9ee51 thru 0x5da196b (0x2b1b bytes) 2024-09-16T22:17:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:3273 write 0x77eaf33 thru 0x77f9ab7 (0xeb85 bytes) 2024-09-16T22:17:05.754 INFO:teuthology.orchestra.run.smithi031.stdout:3276 punch from 0xb7a25f4 to 0xb7ad62d, (0xb039 bytes) 2024-09-16T22:17:05.754 INFO:teuthology.orchestra.run.smithi031.stdout:3277 writesame 0x4246be1 thru 0x4252538 data_size 0x24(0xb958 bytes) 2024-09-16T22:17:05.789 INFO:teuthology.orchestra.run.smithi031.stdout:3280 writesame 0xd8bdbc5 thru 0xd8c47f8 data_size 0x32(0x6c34 bytes) 2024-09-16T22:17:05.799 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.799 INFO:teuthology.orchestra.run.smithi031.stdout:3282 read 0xbb8b3ae thru 0xbb99a4d (0xe6a0 bytes) 2024-09-16T22:17:05.799 INFO:teuthology.orchestra.run.smithi031.stdout:3286 punch from 0xae2b5b7 to 0xae354db, (0x9f24 bytes) 2024-09-16T22:17:05.800 INFO:teuthology.orchestra.run.smithi031.stdout:3287 write 0xd5bd033 thru 0xd5c8b4b (0xbb19 bytes) 2024-09-16T22:17:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:3289 punch from 0xcfc4b35 to 0xcfcd86c, (0x8d37 bytes) 2024-09-16T22:17:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:3290 read 0x6174010 thru 0x617be13 (0x7e04 bytes) 2024-09-16T22:17:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:3291 punch from 0x6d6be4c to 0x6d76ba3, (0xad57 bytes) 2024-09-16T22:17:05.816 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:05.818+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:05.816 INFO:teuthology.orchestra.run.smithi031.stdout:3294 write 0xe92719b thru 0xe92bba3 (0x4a09 bytes) 2024-09-16T22:17:05.827 INFO:teuthology.orchestra.run.smithi031.stdout:3295 punch from 0x55a023 to 0x5613b2, (0x738f bytes) 2024-09-16T22:17:05.827 INFO:teuthology.orchestra.run.smithi031.stdout:3296 read 0x9b4a680 thru 0x9b52bd1 (0x8552 bytes) 2024-09-16T22:17:05.827 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:05.827 INFO:teuthology.orchestra.run.smithi031.stdout:3300 read 0xdc522df thru 0xdc5e885 (0xc5a7 bytes) 2024-09-16T22:17:05.827 INFO:teuthology.orchestra.run.smithi031.stdout:3301 write 0x8995f87 thru 0x89a0407 (0xa481 bytes) 2024-09-16T22:17:05.840 INFO:teuthology.orchestra.run.smithi031.stdout:3303 write 0x6ab02fe thru 0x6abde46 (0xdb49 bytes) 2024-09-16T22:17:05.844 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.844 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.851 INFO:teuthology.orchestra.run.smithi031.stdout:3306 writesame 0xe43e2ab thru 0xe44870c data_size 0x35(0xa462 bytes) 2024-09-16T22:17:05.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:05.860 INFO:teuthology.orchestra.run.smithi031.stdout:3307 read 0xb54a0c0 thru 0xb5536e9 (0x962a bytes) 2024-09-16T22:17:05.860 INFO:teuthology.orchestra.run.smithi031.stdout:3308 read 0x1cdefd thru 0x1d3d81 (0x5e85 bytes) 2024-09-16T22:17:05.861 INFO:teuthology.orchestra.run.smithi031.stdout:3309 writesame 0x6cc73e thru 0x6dada1 data_size 0x1e(0xe664 bytes) 2024-09-16T22:17:05.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:17:05.883+0000 7fe7ced9d640 0 --2- 172.21.15.31:0/1118169203 >> v2:172.21.15.31:3300/0 conn(0x7fe7d00b9170 0x7fe7d0142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T22:17:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:3310 trunc from 0xe92bba4 to 0xccab4f9 2024-09-16T22:17:05.910 INFO:teuthology.orchestra.run.smithi031.stdout:3312 read 0x78e1df thru 0x79c2ee (0xe110 bytes) 2024-09-16T22:17:05.912 INFO:teuthology.orchestra.run.smithi031.stdout:3313 trunc from 0xccab4f9 to 0xd0f1dcc 2024-09-16T22:17:05.917 INFO:teuthology.orchestra.run.smithi031.stdout:3314 writesame 0xa15c130 thru 0xa15e788 data_size 0x1(0x2659 bytes) 2024-09-16T22:17:05.917 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:05.919+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:05.923 INFO:teuthology.orchestra.run.smithi031.stdout:3316 clone 31 order 21 su 65536 sc 6 2024-09-16T22:17:05.979 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:05.979 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:05.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:06.031 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:06.033+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:06.090 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.090 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:06.142 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:06.144+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:clear 3.0 pg_upmap mapping 2024-09-16T22:17:06.231 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:06.233+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:06.235 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone30 from 0xd0f1dcc (overlap 0x63e910a) to 0x0 2024-09-16T22:17:06.261 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.262 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:06.320 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:06.322+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:06.381 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.381 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:06.431 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:06.433+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:06.523 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:06.524+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:06.533 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.533 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:06.623 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:06.624+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:06.645 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.645 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:06.724 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:06.725+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:06.797 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.797 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:06.849 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:06.851+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:06.907 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:06.908 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:06.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:06.926 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:06.928+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:07.027 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:07.028+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:07.058 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:07.058 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:07.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:07.128 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:07.129+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:07.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:07.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:17:07.213 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:07.213 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:07.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:07.232 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:07.233+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:07.323 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:07.325 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:07.325 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:07.329 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:07.331+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:07.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:07.341 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:07.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:07.360 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:07.376 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:07.376 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:07.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:07.409 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:07.426 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:07.429 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:07.431+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:07.439 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:07.440 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:07.445 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:07.445 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:07.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:07.531 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:07.533+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:07.558 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:07.558 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:07.558 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:07.558 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:07.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:07.632 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:07.633+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:07.673 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:07.674 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:07.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:07.766 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:07.768+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:07.831 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:07.831 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:07.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:07.831 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:07.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42088 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 178176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851860013, 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021352987, 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42088 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980192+0000", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9699999999999996e-07 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980194+0000", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980200+0000", 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2580000000000001e-06 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980205+0000", 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6399999999999996e-06 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001414+0000", 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021208751000000001 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001482+0000", 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7868999999999997e-05 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001545+0000", 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2821e-05 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42089 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 186368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851760122999998, 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021459101000000001, 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42089 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980292+0000", 2024-09-16T22:17:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980295+0000", 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980303+0000", 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8490000000000006e-06 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980309+0000", 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.947e-06 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001584+0000", 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021275051999999999 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001666+0000", 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1706999999999995e-05 2024-09-16T22:17:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001751+0000", 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4906000000000003e-05 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42090 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 194560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851733994, 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021544528, 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42090 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980320+0000", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980327+0000", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5649999999999998e-06 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980332+0000", 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-16T22:17:07.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001788+0000", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021456295 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001826+0000", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8083000000000001e-05 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001862+0000", 2024-09-16T22:17:07.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5722999999999997e-05 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42091 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851712788, 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021714943, 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42091 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:07.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980341+0000", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9180000000000001e-06 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980348+0000", 2024-09-16T22:17:07.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980353+0000", 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9339999999999997e-06 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001898+0000", 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021545557 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001979+0000", 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0061999999999993e-05 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002054+0000", 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5160000000000005e-05 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42092 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 210944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:07.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851691815999999, 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021811192, 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42092 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:07.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980369+0000", 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6959999999999994e-06 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980376+0000", 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8789999999999997e-06 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980388+0000", 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1996e-05 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002082+0000", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021694083999999999 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002131+0000", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000003e-05 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002171+0000", 2024-09-16T22:17:07.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9836000000000003e-05 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42093 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 219136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851124887000001, 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021327148000000001, 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42093 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980927+0000", 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-07 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980931+0000", 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4960000000000001e-06 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980942+0000", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0757999999999999e-05 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980950+0000", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-06 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002187+0000", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021237018999999999 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002220+0000", 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2861000000000003e-05 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002254+0000", 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3818e-05 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42094 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 227328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851090734, 2024-09-16T22:17:07.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021427170999999998, 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42094 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980964+0000", 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4150000000000002e-06 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980970+0000", 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6429999999999998e-06 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980984+0000", 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3259e-05 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002268+0000", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021284416 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002325+0000", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002388+0000", 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:17:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42095 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 235520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851059996, 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021555900999999999, 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42095 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:07.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980994+0000", 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981001+0000", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981007+0000", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8189999999999997e-06 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002413+0000", 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021406027000000001 2024-09-16T22:17:07.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002477+0000", 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4243000000000001e-05 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002547+0000", 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9980000000000004e-05 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42096 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 243712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851036805, 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021688049000000001, 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42096 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981017+0000", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981023+0000", 2024-09-16T22:17:07.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981028+0000", 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1370000000000001e-06 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002575+0000", 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021546684999999999 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002637+0000", 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1950000000000001e-05 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002703+0000", 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6268999999999999e-05 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42097 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 251904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.851017009, 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021820512, 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42097 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:07.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981037+0000", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7889999999999999e-06 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981044+0000", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.396e-06 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981049+0000", 2024-09-16T22:17:07.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2669999999999997e-06 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002728+0000", 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021678669000000001 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002787+0000", 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8749000000000003e-05 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002855+0000", 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8162000000000002e-05 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42098 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 260096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.850994277000002, 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021928260000000002, 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42098 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981058+0000", 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981066+0000", 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1710000000000002e-06 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981074+0000", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.746e-06 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981079+0000", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002883+0000", 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021803978000000002 2024-09-16T22:17:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002941+0000", 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8700999999999999e-05 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002986+0000", 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4073000000000003e-05 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42763 3.15 3:ae84599f:::rbd_data.11cd49ca855d.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.449811765, 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024874701999999999, 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42763 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382241+0000", 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5899999999999995e-07 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382245+0000", 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382255+0000", 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9299999999999998e-06 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382260+0000", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382313+0000", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3091999999999999e-05 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382359+0000", 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6866999999999997e-05 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.386616+0000", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042566100000000001 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407035+0000", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020419085 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407047+0000", 2024-09-16T22:17:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1836e-05 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407080+0000", 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2892999999999998e-05 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407114+0000", 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4569000000000001e-05 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59483 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8994433849999997, 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028869210999999999, 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59483 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932611+0000", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8710000000000001e-06 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936104+0000", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034931049999999998 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936128+0000", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3751e-05 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936140+0000", 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2428e-05 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936233+0000", 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2406999999999998e-05 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936414+0000", 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018144400000000001 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.937091+0000", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067658099999999999 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958379+0000", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02128855 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961342+0000", 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002962667 2024-09-16T22:17:07.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961400+0000", 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8335000000000003e-05 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961477+0000", 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7071999999999997e-05 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59484 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [write 0~0] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8928627999999996, 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025486099000000002, 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59484 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939190+0000", 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.716e-06 2024-09-16T22:17:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939198+0000", 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4800000000000004e-06 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939219+0000", 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0851000000000001e-05 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939229+0000", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0599e-05 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947061+0000", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078311649999999993 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947144+0000", 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3531999999999999e-05 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947346+0000", 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020164299999999999 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958658+0000", 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011312467 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964560+0000", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059020119999999999 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964626+0000", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6025999999999994e-05 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964675+0000", 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8607999999999999e-05 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59481 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'145, mlcod=59'143)", 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8919657430000001, 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025627553000000001, 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940088+0000", 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1660000000000001e-06 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945430+0000", 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053420330000000004 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945444+0000", 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3787e-05 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945453+0000", 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945526+0000", 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3166999999999999e-05 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945549+0000", 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3666e-05 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965653+0000", 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020104052000000001 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965713+0000", 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9960999999999997e-05 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59485 3.1d e60/23 3:bcb688ae:::rbd_data.11cdc742ac9f.0000000000000002:head v 60'169, mlcod=59'167)", 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.891183989, 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025808811000000001, 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:07.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8400000000000003e-07 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944623+0000", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003754949 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944642+0000", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8539999999999999e-05 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944654+0000", 2024-09-16T22:17:07.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1905e-05 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944767+0000", 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113485 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944793+0000", 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5459000000000001e-05 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966625+0000", 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021832767999999999 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966676+0000", 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0921000000000003e-05 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59482 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'147, mlcod=59'143)", 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.8865605359999993, 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021422455999999999, 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945492+0000", 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.64e-07 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945500+0000", 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6260000000000003e-06 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945506+0000", 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4910000000000002e-06 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945511+0000", 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951314+0000", 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058029509999999998 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951341+0000", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7387000000000001e-05 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966873+0000", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015531978 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966913+0000", 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:64678 3.f 3:f2f24fe2:::rbd_data.11cdb2b6d61b.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.5473253719999995, 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025102092999999999, 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 64678 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284728+0000", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.381e-06 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284733+0000", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5509999999999999e-06 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284744+0000", 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294999999999999e-05 2024-09-16T22:17:07.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284750+0000", 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284795+0000", 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5697e-05 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284893+0000", 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7775000000000005e-05 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.288943+0000", 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040499730000000001 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309744+0000", 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020800511000000001 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309757+0000", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3165e-05 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309793+0000", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6183999999999998e-05 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309828+0000", 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5326e-05 2024-09-16T22:17:07.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:67349 3.3 3:c2f8d2bb:::rbd_data.11cd67c2f996.0000000000000000:1f [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3996558889999999, 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021461905999999999, 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 67349 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432398+0000", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7859999999999999e-06 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432404+0000", 2024-09-16T22:17:07.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432409+0000", 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9570000000000001e-06 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453687+0000", 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021277653000000001 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453762+0000", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5123000000000006e-05 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453858+0000", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5699000000000006e-05 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:67350 3.3 3:c2f8d2bb:::rbd_data.11cd67c2f996.0000000000000000:1f [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3996361679999998, 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021673142999999999, 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 67350 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432416+0000", 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432422+0000", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0020000000000003e-06 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432428+0000", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0589999999999999e-06 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432433+0000", 2024-09-16T22:17:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1909999999999999e-06 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453887+0000", 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021454253 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453977+0000", 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9647000000000004e-05 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.454089+0000", 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011156500000000001 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:07.930 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:07.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:07.932 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:07.933+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:08.032 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:08.034+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:08.079 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:08.080 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:08.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:08.133 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:08.135+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:08.189 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:17:08.234 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:08.235+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:08.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:08.335 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:08.336+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:08.436 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:08.438+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55222 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4317184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.103923158000001, 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063938485000000003, 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55222 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336759+0000", 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000208063 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336764+0000", 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336776+0000", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.182e-05 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336782+0000", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7789999999999998e-06 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400086+0000", 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063303645000000006 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400310+0000", 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022399299999999999 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400489+0000", 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017931300000000001 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55223 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.103679825, 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911932000000005, 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55223 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336950+0000", 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155821 2024-09-16T22:17:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336954+0000", 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336965+0000", 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1055e-05 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336972+0000", 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4309999999999999e-06 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400531+0000", 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559177999999994 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400612+0000", 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1360999999999996e-05 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400706+0000", 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3837000000000003e-05 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55224 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4333568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.103483718, 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064193369, 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55224 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337203+0000", 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021318899999999999 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337208+0000", 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1710000000000004e-06 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337215+0000", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3089999999999998e-06 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337219+0000", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400749+0000", 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063530160000000002 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401000+0000", 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000250995 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401184+0000", 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018346399999999999 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55225 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4919296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.103246907000001, 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064167392000000004, 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55225 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337451+0000", 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022388599999999999 2024-09-16T22:17:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337456+0000", 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7280000000000003e-06 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337466+0000", 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0351e-05 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337473+0000", 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7449999999999998e-06 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401218+0000", 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063745183999999996 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401298+0000", 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0302999999999993e-05 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401394+0000", 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6194999999999996e-05 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55226 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4349952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.102989751000001, 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064121972999999999, 2024-09-16T22:17:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55226 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337709+0000", 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022441200000000001 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337713+0000", 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337724+0000", 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0740999999999999e-05 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337731+0000", 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:17:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401426+0000", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063695389000000005 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401507+0000", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0356000000000001e-05 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401606+0000", 2024-09-16T22:17:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9631999999999997e-05 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55227 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4935680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.102731010999999, 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064076279, 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55227 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337889+0000", 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146055 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337894+0000", 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337901+0000", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337905+0000", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1080000000000003e-06 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401637+0000", 2024-09-16T22:17:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063731333000000001 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401721+0000", 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3969999999999997e-05 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401819+0000", 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55228 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4366336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.102560559, 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064169421000000004, 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55228 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338139+0000", 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022557499999999999 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338143+0000", 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-09-16T22:17:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338158+0000", 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4498000000000001e-05 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338165+0000", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401880+0000", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063715331 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401978+0000", 2024-09-16T22:17:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8158000000000004e-05 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402083+0000", 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104799 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55229 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.102221633999999, 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064053115999999993, 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55229 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338426+0000", 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017367500000000001 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338431+0000", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338442+0000", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1749999999999999e-05 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338449+0000", 2024-09-16T22:17:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6649999999999997e-06 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402117+0000", 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063668279999999994 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402207+0000", 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9257000000000003e-05 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402305+0000", 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8911000000000002e-05 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55231 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4968448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101643444, 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911281, 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55231 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-07 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338834+0000", 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338841+0000", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338847+0000", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1360000000000002e-06 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402566+0000", 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063718876999999993 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402647+0000", 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1384000000000005e-05 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402742+0000", 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4686999999999997e-05 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55232 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4399104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101618441999999, 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064122381000000006, 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55232 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339005+0000", 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014984300000000001 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339010+0000", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999999e-06 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339023+0000", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2904e-05 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339030+0000", 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402777+0000", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063746612999999994 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402864+0000", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7818000000000005e-05 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402978+0000", 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113507 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55233 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4984832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101432495999999, 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151505999999997, 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55233 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339210+0000", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016822699999999999 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.741e-06 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339225+0000", 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0149000000000001e-05 2024-09-16T22:17:08.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339232+0000", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403011+0000", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063779340000000004 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403096+0000", 2024-09-16T22:17:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5482000000000006e-05 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403193+0000", 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6496999999999994e-05 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55234 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101226133000001, 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151969000000003, 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55234 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339412+0000", 2024-09-16T22:17:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000164544 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339417+0000", 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6160000000000004e-06 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339428+0000", 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0522e-05 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339435+0000", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7880000000000007e-06 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403227+0000", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063791688999999999 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403309+0000", 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2254000000000002e-05 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403400+0000", 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0556000000000005e-05 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55235 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5001216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101026539999999, 2024-09-16T22:17:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064168931999999998, 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55235 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339684+0000", 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023680199999999999 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339690+0000", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.049e-06 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339701+0000", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0494e-05 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339706+0000", 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6579999999999999e-06 2024-09-16T22:17:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403431+0000", 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063724905999999998 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403519+0000", 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7892999999999993e-05 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403616+0000", 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.713e-05 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55236 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4431872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.100756884000001, 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064115839999999993, 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55236 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339965+0000", 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024771199999999998 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339970+0000", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339980+0000", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0254999999999999e-05 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339987+0000", 2024-09-16T22:17:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7519999999999999e-06 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403650+0000", 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063663553999999997 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403736+0000", 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5778000000000004e-05 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403833+0000", 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6854000000000005e-05 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55237 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5017600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.100474923, 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064058452000000002, 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55237 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340000+0000", 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340002+0000", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5859999999999999e-06 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340009+0000", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2080000000000003e-06 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340015+0000", 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403865+0000", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063850812000000007 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403955+0000", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9648999999999995e-05 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404057+0000", 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102472 2024-09-16T22:17:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55238 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4448256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.100450345, 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064252408999999996, 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55238 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340290+0000", 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026611899999999999 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340294+0000", 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2810000000000002e-06 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340302+0000", 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4630000000000001e-06 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340308+0000", 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8259999999999998e-06 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404092+0000", 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063783230999999996 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404178+0000", 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6080000000000003e-05 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404276+0000", 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8408999999999999e-05 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55239 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5033984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.100153794000001, 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064172565000000001, 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55239 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341020+0000", 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070000599999999998 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341025+0000", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6589999999999998e-06 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341038+0000", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3534e-05 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341046+0000", 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6179999999999997e-06 2024-09-16T22:17:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404308+0000", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063261957999999993 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404393+0000", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4791999999999997e-05 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404493+0000", 2024-09-16T22:17:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9998e-05 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55243 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5066752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.099290330000001, 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064032600999999995, 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55243 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341467+0000", 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028289899999999998 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341471+0000", 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5569999999999997e-06 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341481+0000", 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0339e-05 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341488+0000", 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405047+0000", 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559063999999998 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405129+0000", 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1919e-05 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405216+0000", 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7230000000000003e-05 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55244 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4497408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.098974643, 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063903336000000005, 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55244 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341673+0000", 2024-09-16T22:17:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017377700000000001 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341678+0000", 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341688+0000", 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0200000000000001e-05 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341695+0000", 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.55e-06 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405249+0000", 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063554352999999994 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405323+0000", 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3677000000000006e-05 2024-09-16T22:17:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405403+0000", 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9628000000000004e-05 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55245 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5083136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.098765539, 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063879072999999995, 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55245 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341930+0000", 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022177199999999999 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341940+0000", 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341950+0000", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0124e-05 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341959+0000", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7260000000000008e-06 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405436+0000", 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063477512 2024-09-16T22:17:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405509+0000", 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2156999999999994e-05 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405587+0000", 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8957999999999999e-05 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:08.535 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:08.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:08.578 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:08.579+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:08.640 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:08.641+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:08.729 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:08.729 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:08.729 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:08.729 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:08.739 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:08.741+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:08.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:08.841 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:08.842+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:08.935 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:08.936 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:08.941 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:08.943+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:08.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:09.042 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:09.043+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:09.054 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T22:17:09.077 INFO:teuthology.orchestra.run.smithi031.stdout:3317 read 0x32c939e thru 0x32ca8d2 (0x1535 bytes) 2024-09-16T22:17:09.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:09.079 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:09.079 INFO:teuthology.orchestra.run.smithi031.stdout:3320 write 0x21c5fe8 thru 0x21cdc8a (0x7ca3 bytes) 2024-09-16T22:17:09.110 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.534711784999999, 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.534692445000001, 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.534288045, 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:09.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:09.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45048 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.532614599999999, 2024-09-16T22:17:09.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676787000000002, 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45048 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:09.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579108+0000", 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579110+0000", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579116+0000", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579120+0000", 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:17:09.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630608+0000", 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051487796000000002 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630687+0000", 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8709999999999997e-05 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630784+0000", 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7190999999999998e-05 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.532592775000001, 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:09.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.532562242000001, 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:09.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:09.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.528161343000001, 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:09.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:09.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.528140862000001, 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:09.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.528120286, 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.528095800999999, 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:09.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.528074230000001, 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:09.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:09.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:09.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45147 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.522016455999999, 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051674649000000003, 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45147 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589706+0000", 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:09.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589708+0000", 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9599999999999999e-06 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589714+0000", 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589720+0000", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.383e-06 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641214+0000", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051494686999999997 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641285+0000", 2024-09-16T22:17:09.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0795000000000002e-05 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641380+0000", 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5118999999999995e-05 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521995631999999, 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:09.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:09.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521975300000001, 2024-09-16T22:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:09.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521954081000001, 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:09.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521932709000001, 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:09.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:09.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521911871, 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:09.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:09.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521891289999999, 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:09.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.521869723000002, 2024-09-16T22:17:09.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:09.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.517222848999999, 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:09.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:09.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:09.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:09.205 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:09.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:09.207 INFO:teuthology.orchestra.run.smithi031.stdout:3321 clone 32 order 25 su 262144 sc 14 2024-09-16T22:17:09.207 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:09.208+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:09.307 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:09.309+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:09.354 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:09.355 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:09.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:09.408 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:09.411+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:09.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:09.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:09.474 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:09.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:09.510 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:09.512+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:09.583 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:09.600 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:09.611 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:09.613+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:09.617 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:09.634 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:09.634 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:09.634 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:09.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:09.651 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:09.651 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:09.668 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:09.685 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:09.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:09.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:09.712 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:09.713+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:09.747 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:09.747 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:09.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:09.813 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:09.815+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:09.858 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:09.858 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:09.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:09.914 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:09.916+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:09.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:10.019 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:10.021+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:10.115 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:10.117+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:10.121 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:10.121 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:10.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:10.193 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone31 from 0xd0f1dcc (overlap 0xd0f1dcc) to 0x14e4fc7 2024-09-16T22:17:10.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:17:10.249 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:10.251+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:10.316 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:10.318+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:10.335 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:10.335 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:10.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:10.417 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:10.419+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:10.518 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:10.520+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:10.541 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:10.541 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:10.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:10.619 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:10.621+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:10.720 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:10.722+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:10.752 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:10.752 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:10.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:10.847 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:10.849+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:10.922 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:10.924+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:10.940 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:10.940 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:10.940 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T22:17:10.941 INFO:teuthology.orchestra.run.smithi031.stdout:3322 read 0x3de50a thru 0x3e31be (0x4cb5 bytes) 2024-09-16T22:17:10.943 INFO:teuthology.orchestra.run.smithi031.stdout:3323 punch from 0x3f9d053 to 0x3facce4, (0xfc91 bytes) 2024-09-16T22:17:10.946 INFO:teuthology.orchestra.run.smithi031.stdout:3324 punch from 0x1010759 to 0x101bf54, (0xb7fb bytes) 2024-09-16T22:17:10.946 INFO:teuthology.orchestra.run.smithi031.stdout:3325 punch from 0xac72aca to 0xac7f25f, (0xc795 bytes) 2024-09-16T22:17:10.946 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:10.946 INFO:teuthology.orchestra.run.smithi031.stdout:3327 read 0x34f7434 thru 0x34fb78f (0x435c bytes) 2024-09-16T22:17:10.947 INFO:teuthology.orchestra.run.smithi031.stdout:3328 writesame 0x2882b81 thru 0x2888939 data_size 0x1(0x5db9 bytes) 2024-09-16T22:17:10.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:11.034 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:11.036+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:3330 write 0x3254a09 thru 0x325eb61 (0xa159 bytes) 2024-09-16T22:17:11.117 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:11.117 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:11.124 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:11.126+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:11.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:11.170 INFO:teuthology.orchestra.run.smithi031.stdout:3331 write 0x84ab1fc thru 0x84b5ecc (0xacd1 bytes) 2024-09-16T22:17:11.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:11.191 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:11.191 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T22:17:11.191 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:11.191 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:11.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-16T22:17:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:17:11.219 INFO:teuthology.orchestra.run.smithi031.stdout:3332 write 0x34813f5 thru 0x348774a (0x6356 bytes) 2024-09-16T22:17:11.253 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:11.254+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:11.309 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:11.309 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:11.315 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:11.315 INFO:teuthology.orchestra.run.smithi031.stdout:3334 read 0xc299346 thru 0xc2a788a (0xe545 bytes) 2024-09-16T22:17:11.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:11.316 INFO:teuthology.orchestra.run.smithi031.stdout:3338 read 0xbe7dce6 thru 0xbe8c13f (0xe45a bytes) 2024-09-16T22:17:11.316 INFO:teuthology.orchestra.run.smithi031.stdout:3339 writesame 0xbf45f83 thru 0xbf4c559 data_size 0x1d(0x65d7 bytes) 2024-09-16T22:17:11.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:11.327 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:11.328+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:11.354 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:17:11.354 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:11.408 INFO:teuthology.orchestra.run.smithi031.stdout:3340 write 0xa5984d thru 0xa5be62 (0x2616 bytes) 2024-09-16T22:17:11.428 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:11.428+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:11.472 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:11.472 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:11.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:11.502 INFO:teuthology.orchestra.run.smithi031.stdout:3342 read 0xc04f428 thru 0xc05676e (0x7347 bytes) 2024-09-16T22:17:11.507 INFO:teuthology.orchestra.run.smithi031.stdout:3344 trunc from 0xd0f1dcc to 0xa642cfe 2024-09-16T22:17:11.528 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:11.529+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:11.630 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:11.631+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:11.707 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:11.707 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:11.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:11.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:11.760 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:11.762+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:11.831 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:11.832+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:11.884 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:11.884 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:11.903 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:11.903 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:11.903 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:11.903 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:11.904 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:11.904 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:11.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:11.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:11.921 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:11.932 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:11.934+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:11.937 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:11.938 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:11.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:11.972 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:11.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:12.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:12.007 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:12.030 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:12.030 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:12.032 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:12.034+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:12.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:12.129 INFO:teuthology.orchestra.run.smithi031.stdout:3345 trunc from 0xa642cfe to 0x9f16afd 2024-09-16T22:17:12.133 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:12.135+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:12.143 INFO:teuthology.orchestra.run.smithi031.stdout:3347 write 0x8b5db3b thru 0x8b62edf (0x53a5 bytes) 2024-09-16T22:17:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:3349 punch from 0xa30379 to 0xa304e3, (0x16a bytes) 2024-09-16T22:17:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:3350 punch from 0x9c55364 to 0x9c5cb3a, (0x77d6 bytes) 2024-09-16T22:17:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:3351 read 0x6853fe2 thru 0x6854df3 (0xe12 bytes) 2024-09-16T22:17:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:3354 read 0x70c254f thru 0x70cb1f8 (0x8caa bytes) 2024-09-16T22:17:12.146 INFO:teuthology.orchestra.run.smithi031.stdout:3355 read 0x34b274b thru 0x34b8f04 (0x67ba bytes) 2024-09-16T22:17:12.146 INFO:teuthology.orchestra.run.smithi031.stdout:3356 read 0x5429aaf thru 0x542b437 (0x1989 bytes) 2024-09-16T22:17:12.154 INFO:teuthology.orchestra.run.smithi031.stdout:3358 writesame 0xb6551fc thru 0xb65804d data_size 0x16(0x2e52 bytes) 2024-09-16T22:17:12.156 INFO:teuthology.orchestra.run.smithi031.stdout:3359 punch from 0x950c447 to 0x951472c, (0x82e5 bytes) 2024-09-16T22:17:12.156 INFO:teuthology.orchestra.run.smithi031.stdout:3360 read 0x6932739 thru 0x693ffe8 (0xd8b0 bytes) 2024-09-16T22:17:12.156 INFO:teuthology.orchestra.run.smithi031.stdout:3361 read 0x2168387 thru 0x2176987 (0xe601 bytes) 2024-09-16T22:17:12.157 INFO:teuthology.orchestra.run.smithi031.stdout:3362 write 0x42265d2 thru 0x42300dc (0x9b0b bytes) 2024-09-16T22:17:12.169 INFO:teuthology.orchestra.run.smithi031.stdout:3363 write 0xe5b6046 thru 0xe5c37a6 (0xd761 bytes) 2024-09-16T22:17:12.170 INFO:teuthology.orchestra.run.smithi031.stdout:3364 write 0xabae15e thru 0xabb02fe (0x21a1 bytes) 2024-09-16T22:17:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:3365 punch from 0x628b8f8 to 0x6295ef6, (0xa5fe bytes) 2024-09-16T22:17:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:3366 write 0x33959c7 thru 0x339b63d (0x5c77 bytes) 2024-09-16T22:17:12.174 INFO:teuthology.orchestra.run.smithi031.stdout:3367 write 0xb22558e thru 0xb22575c (0x1cf bytes) 2024-09-16T22:17:12.176 INFO:teuthology.orchestra.run.smithi031.stdout:3368 writesame 0x45a1df thru 0x461f7a data_size 0x2(0x7d9c bytes) 2024-09-16T22:17:12.184 INFO:teuthology.orchestra.run.smithi031.stdout:3369 read 0x31bb212 thru 0x31bc7b0 (0x159f bytes) 2024-09-16T22:17:12.185 INFO:teuthology.orchestra.run.smithi031.stdout:3370 write 0x1213b9 thru 0x1258ea (0x4532 bytes) 2024-09-16T22:17:12.187 INFO:teuthology.orchestra.run.smithi031.stdout:3373 writesame 0x93be12e thru 0x93cb500 data_size 0x1(0xd3d3 bytes) 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42089 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 186368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.226186195, 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021459101000000001, 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42089 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980291+0000", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980292+0000", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980295+0000", 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-16T22:17:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980303+0000", 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8490000000000006e-06 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980309+0000", 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.947e-06 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001584+0000", 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021275051999999999 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001666+0000", 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1706999999999995e-05 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001751+0000", 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4906000000000003e-05 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42090 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 194560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.226160065999998, 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021544528, 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42090 2024-09-16T22:17:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980318+0000", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980320+0000", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980327+0000", 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5649999999999998e-06 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980332+0000", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001788+0000", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021456295 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001826+0000", 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8083000000000001e-05 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001862+0000", 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5722999999999997e-05 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42091 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.226138859999999, 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021714943, 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42091 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980339+0000", 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-16T22:17:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980341+0000", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9180000000000001e-06 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980348+0000", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980353+0000", 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9339999999999997e-06 2024-09-16T22:17:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001898+0000", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021545557 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.001979+0000", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0061999999999993e-05 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002054+0000", 2024-09-16T22:17:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5160000000000005e-05 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42092 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 210944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.226117888000001, 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021811192, 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42092 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980360+0000", 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980369+0000", 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6959999999999994e-06 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980376+0000", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8789999999999997e-06 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980388+0000", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1996e-05 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002082+0000", 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021694083999999999 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002131+0000", 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000003e-05 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002171+0000", 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9836000000000003e-05 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42094 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 227328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.225516806000002, 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021427170999999998, 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42094 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980961+0000", 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980964+0000", 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4150000000000002e-06 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980970+0000", 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6429999999999998e-06 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980984+0000", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3259e-05 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002268+0000", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021284416 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002325+0000", 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002388+0000", 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3176999999999995e-05 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42095 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 235520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.225486067999999, 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021555900999999999, 2024-09-16T22:17:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42095 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980992+0000", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.980994+0000", 2024-09-16T22:17:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981001+0000", 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981007+0000", 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8189999999999997e-06 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002413+0000", 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021406027000000001 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002477+0000", 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4243000000000001e-05 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002547+0000", 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9980000000000004e-05 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42096 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 243712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.225462876999998, 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021688049000000001, 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42096 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981015+0000", 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981017+0000", 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981023+0000", 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981028+0000", 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1370000000000001e-06 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002575+0000", 2024-09-16T22:17:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021546684999999999 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002637+0000", 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1950000000000001e-05 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002703+0000", 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6268999999999999e-05 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42097 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 251904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.225443081000002, 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021820512, 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42097 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981035+0000", 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981037+0000", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7889999999999999e-06 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981044+0000", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.396e-06 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981049+0000", 2024-09-16T22:17:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2669999999999997e-06 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002728+0000", 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021678669000000001 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002787+0000", 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8749000000000003e-05 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002855+0000", 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8162000000000002e-05 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42098 3.f 3:f579190b:::rbd_data.11cd43ed039f.000000000000011b:16 [read 260096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.225420349, 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021928260000000002, 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42098 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981057+0000", 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981058+0000", 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981066+0000", 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1710000000000002e-06 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981074+0000", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.746e-06 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:48.981079+0000", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002883+0000", 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021803978000000002 2024-09-16T22:17:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002941+0000", 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8700999999999999e-05 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.002986+0000", 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4073000000000003e-05 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:42763 3.15 3:ae84599f:::rbd_data.11cd49ca855d.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.824237836999998, 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024874701999999999, 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 42763 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382240+0000", 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382241+0000", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5899999999999995e-07 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382245+0000", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0720000000000001e-06 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382255+0000", 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9299999999999998e-06 2024-09-16T22:17:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382260+0000", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382313+0000", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3091999999999999e-05 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.382359+0000", 2024-09-16T22:17:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6866999999999997e-05 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.386616+0000", 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042566100000000001 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407035+0000", 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020419085 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407047+0000", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1836e-05 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407080+0000", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2892999999999998e-05 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:49.407114+0000", 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4569000000000001e-05 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59483 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.273869457, 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028869210999999999, 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59483 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932608+0000", 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.932611+0000", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8710000000000001e-06 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936104+0000", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034931049999999998 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936128+0000", 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3751e-05 2024-09-16T22:17:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936140+0000", 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2428e-05 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936233+0000", 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2406999999999998e-05 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.936414+0000", 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018144400000000001 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.937091+0000", 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067658099999999999 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958379+0000", 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02128855 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961342+0000", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002962667 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961400+0000", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8335000000000003e-05 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.961477+0000", 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7071999999999997e-05 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:59484 3.c 3:32efde54:::rbd_data.11cdc742ac9f.0000000000000001:head [write 0~0] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.267288872, 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025486099000000002, 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 59484 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939189+0000", 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939190+0000", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.716e-06 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939198+0000", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4800000000000004e-06 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939219+0000", 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0851000000000001e-05 2024-09-16T22:17:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.939229+0000", 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0599e-05 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947061+0000", 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078311649999999993 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947144+0000", 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3531999999999999e-05 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.947346+0000", 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020164299999999999 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.958658+0000", 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011312467 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964560+0000", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059020119999999999 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964626+0000", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6025999999999994e-05 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.964675+0000", 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8607999999999999e-05 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59481 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'145, mlcod=59'143)", 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.266391815, 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025627553000000001, 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940086+0000", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940088+0000", 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1660000000000001e-06 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945430+0000", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053420330000000004 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945444+0000", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3787e-05 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945453+0000", 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945526+0000", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3166999999999999e-05 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945549+0000", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3666e-05 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965653+0000", 2024-09-16T22:17:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020104052000000001 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.965713+0000", 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9960999999999997e-05 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59485 3.1d e60/23 3:bcb688ae:::rbd_data.11cdc742ac9f.0000000000000002:head v 60'169, mlcod=59'167)", 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.265610061, 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025808811000000001, 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.940868+0000", 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8400000000000003e-07 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944623+0000", 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003754949 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944642+0000", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8539999999999999e-05 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944654+0000", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1905e-05 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944767+0000", 2024-09-16T22:17:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113485 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.944793+0000", 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5459000000000001e-05 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966625+0000", 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021832767999999999 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966676+0000", 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0921000000000003e-05 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:59482 3.0 e60/58 3:0161706a:::rbd_data.11cdc742ac9f.0000000000000000:head v 60'147, mlcod=59'143)", 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.260986608, 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021422455999999999, 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945491+0000", 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945492+0000", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.64e-07 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945500+0000", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6260000000000003e-06 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945506+0000", 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4910000000000002e-06 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.945511+0000", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951314+0000", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058029509999999998 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.951341+0000", 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7387000000000001e-05 2024-09-16T22:17:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966873+0000", 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015531978 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.966913+0000", 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:64678 3.f 3:f2f24fe2:::rbd_data.11cdb2b6d61b.000000000000000e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.921751444, 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025102092999999999, 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 64678 2024-09-16T22:17:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284726+0000", 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284728+0000", 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.381e-06 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284733+0000", 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5509999999999999e-06 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284744+0000", 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294999999999999e-05 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284750+0000", 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284795+0000", 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5697e-05 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.284893+0000", 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7775000000000005e-05 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.288943+0000", 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040499730000000001 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309744+0000", 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020800511000000001 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309757+0000", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3165e-05 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309793+0000", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6183999999999998e-05 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:00.309828+0000", 2024-09-16T22:17:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5326e-05 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:67349 3.3 3:c2f8d2bb:::rbd_data.11cd67c2f996.0000000000000000:1f [sparse-read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7740819610000003, 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021461905999999999, 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 67349 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432396+0000", 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432398+0000", 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7859999999999999e-06 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432404+0000", 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432409+0000", 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9570000000000001e-06 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453687+0000", 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021277653000000001 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453762+0000", 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5123000000000006e-05 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453858+0000", 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5699000000000006e-05 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:67350 3.3 3:c2f8d2bb:::rbd_data.11cd67c2f996.0000000000000000:1f [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7740622400000001, 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021673142999999999, 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 67350 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432415+0000", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432416+0000", 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432422+0000", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0020000000000003e-06 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432428+0000", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0589999999999999e-06 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.432433+0000", 2024-09-16T22:17:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1909999999999999e-06 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453887+0000", 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021454253 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.453977+0000", 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9647000000000004e-05 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:04.454089+0000", 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011156500000000001 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14441775900000001, 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115259 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.12389401899999999, 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043168221, 2024-09-16T22:17:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115259 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082587+0000", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087294+0000", 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047076490000000004 2024-09-16T22:17:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087309+0000", 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4551e-05 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087318+0000", 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087372+0000", 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4184999999999997e-05 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087492+0000", 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119762 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087887+0000", 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039564899999999998 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.124754+0000", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036866492000000001 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125690+0000", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093677799999999998 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125726+0000", 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5160999999999998e-05 2024-09-16T22:17:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125752+0000", 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6177e-05 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.310 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:12.311 INFO:teuthology.orchestra.run.smithi031.stdout:3375 writesame 0xb9a5b6b thru 0xb9a8bd0 data_size 0x2a(0x3066 bytes) 2024-09-16T22:17:12.311 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.311 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.311 INFO:teuthology.orchestra.run.smithi031.stdout:3379 trunc from 0xe5c37a7 to 0x5cbaf7c 2024-09-16T22:17:12.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3380 write 0xbe85ce2 thru 0xbe93ef9 (0xe218 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3382 punch from 0x21cf90c to 0x21dd630, (0xdd24 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3383 read 0x987e145 thru 0x9888c1b (0xaad7 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3384 read 0x18c766e thru 0x18cbce2 (0x4675 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3385 write 0x6e1b98f thru 0x6e1ca66 (0x10d8 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3386 write 0xe06949b thru 0xe07153f (0x80a5 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3387 read 0x4044d8d thru 0x404690d (0x1b81 bytes) 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3388 trunc from 0xe071540 to 0x322f01 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.314 INFO:teuthology.orchestra.run.smithi031.stdout:3392 trunc from 0x322f01 to 0xb58168f 2024-09-16T22:17:12.315 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:12.313+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:12.315 INFO:teuthology.orchestra.run.smithi031.stdout:3393 read 0x7126aa9 thru 0x712ed5c (0x82b4 bytes) 2024-09-16T22:17:12.315 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.315 INFO:teuthology.orchestra.run.smithi031.stdout:3395 read 0x9389fa9 thru 0x9398824 (0xe87c bytes) 2024-09-16T22:17:12.315 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.316 INFO:teuthology.orchestra.run.smithi031.stdout:3399 read 0xb50ad15 thru 0xb511a20 (0x6d0c bytes) 2024-09-16T22:17:12.321 INFO:teuthology.orchestra.run.smithi031.stdout:3401 trunc from 0xb58168f to 0x3ebb1ed 2024-09-16T22:17:12.330 INFO:teuthology.orchestra.run.smithi031.stdout:3402 trunc from 0x3ebb1ed to 0x3865215 2024-09-16T22:17:12.351 INFO:teuthology.orchestra.run.smithi031.stdout:3403 trunc from 0x3865215 to 0xe64835b 2024-09-16T22:17:12.357 INFO:teuthology.orchestra.run.smithi031.stdout:3404 read 0xa57c4e6 thru 0xa580317 (0x3e32 bytes) 2024-09-16T22:17:12.358 INFO:teuthology.orchestra.run.smithi031.stdout:3405 read 0x983975e thru 0x983e527 (0x4dca bytes) 2024-09-16T22:17:12.358 INFO:teuthology.orchestra.run.smithi031.stdout:3406 punch from 0xe261b52 to 0xe265cd1, (0x417f bytes) 2024-09-16T22:17:12.358 INFO:teuthology.orchestra.run.smithi031.stdout:3407 writesame 0xd661c01 thru 0xd67161d data_size 0x7(0xfa1d bytes) 2024-09-16T22:17:12.364 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:12.364 INFO:teuthology.orchestra.run.smithi031.stdout:3409 read 0xc923562 thru 0xc92945a (0x5ef9 bytes) 2024-09-16T22:17:12.364 INFO:teuthology.orchestra.run.smithi031.stdout:3410 punch from 0x28ba669 to 0x28c12af, (0x6c46 bytes) 2024-09-16T22:17:12.377 INFO:teuthology.orchestra.run.smithi031.stdout:3411 trunc from 0xe64835b to 0x35c7ed9 2024-09-16T22:17:12.386 INFO:teuthology.orchestra.run.smithi031.stdout:3413 read 0x32b2e6f thru 0x32bbedd (0x906f bytes) 2024-09-16T22:17:12.386 INFO:teuthology.orchestra.run.smithi031.stdout:3414 read 0x3f70ea thru 0x3fc45d (0x5374 bytes) 2024-09-16T22:17:12.388 INFO:teuthology.orchestra.run.smithi031.stdout:3415 trunc from 0x35c7ed9 to 0x52d7682 2024-09-16T22:17:12.411 INFO:teuthology.orchestra.run.smithi031.stdout:3416 writesame 0xea2f7d2 thru 0xea3ed13 data_size 0x1(0xf542 bytes) 2024-09-16T22:17:12.412 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:12.413+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:12.413 INFO:teuthology.orchestra.run.smithi031.stdout:3418 read 0xb7c6a3d thru 0xb7d5284 (0xe848 bytes) 2024-09-16T22:17:12.413 INFO:teuthology.orchestra.run.smithi031.stdout:3419 writesame 0x574a74a thru 0x575028a data_size 0xd(0x5b41 bytes) 2024-09-16T22:17:12.417 INFO:teuthology.orchestra.run.smithi031.stdout:3420 punch from 0xb352cb6 to 0xb357341, (0x468b bytes) 2024-09-16T22:17:12.417 INFO:teuthology.orchestra.run.smithi031.stdout:3421 read 0xa8cbee9 thru 0xa8d7919 (0xba31 bytes) 2024-09-16T22:17:12.417 INFO:teuthology.orchestra.run.smithi031.stdout:3422 read 0xaef5996 thru 0xaefc22c (0x6897 bytes) 2024-09-16T22:17:12.433 INFO:teuthology.orchestra.run.smithi031.stdout:3424 trunc from 0xea3ed14 to 0x13813e7 2024-09-16T22:17:12.442 INFO:teuthology.orchestra.run.smithi031.stdout:3425 read 0x51ae64 thru 0x51c193 (0x1330 bytes) 2024-09-16T22:17:12.442 INFO:teuthology.orchestra.run.smithi031.stdout:3426 clone 33 order 18 su 4096 sc 12 2024-09-16T22:17:12.448 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:12.448 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:12.448 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:12.448 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:12.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:12.541 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:12.543+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:12.604 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:12.605 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:12.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:12.655 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:12.656+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:12.715 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:12.718+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55222 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4317184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.426581154000001, 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063938485000000003, 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55222 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336759+0000", 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000208063 2024-09-16T22:17:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336764+0000", 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336776+0000", 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.182e-05 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336782+0000", 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7789999999999998e-06 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400086+0000", 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063303645000000006 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400310+0000", 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022399299999999999 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400489+0000", 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017931300000000001 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55223 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.426337821000001, 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911932000000005, 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55223 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336950+0000", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155821 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336954+0000", 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:17:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336965+0000", 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1055e-05 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336972+0000", 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4309999999999999e-06 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400531+0000", 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559177999999994 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400612+0000", 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1360999999999996e-05 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400706+0000", 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3837000000000003e-05 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55224 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4333568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.426141714, 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064193369, 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55224 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337203+0000", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021318899999999999 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337208+0000", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1710000000000004e-06 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337215+0000", 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3089999999999998e-06 2024-09-16T22:17:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337219+0000", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400749+0000", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063530160000000002 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401000+0000", 2024-09-16T22:17:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000250995 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401184+0000", 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018346399999999999 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55225 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4919296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.425904902999999, 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064167392000000004, 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55225 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337451+0000", 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022388599999999999 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337456+0000", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7280000000000003e-06 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337466+0000", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0351e-05 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337473+0000", 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7449999999999998e-06 2024-09-16T22:17:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401218+0000", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063745183999999996 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401298+0000", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0302999999999993e-05 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401394+0000", 2024-09-16T22:17:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6194999999999996e-05 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55226 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4349952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.425647746999999, 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064121972999999999, 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55226 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337709+0000", 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022441200000000001 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337713+0000", 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337724+0000", 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0740999999999999e-05 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337731+0000", 2024-09-16T22:17:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401426+0000", 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063695389000000005 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401507+0000", 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0356000000000001e-05 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401606+0000", 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9631999999999997e-05 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55227 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4935680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.425389007, 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064076279, 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55227 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337889+0000", 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146055 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337894+0000", 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337901+0000", 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337905+0000", 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1080000000000003e-06 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401637+0000", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063731333000000001 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401721+0000", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3969999999999997e-05 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401819+0000", 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:17:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55228 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4366336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.425218555000001, 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064169421000000004, 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55228 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338139+0000", 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022557499999999999 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338143+0000", 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338158+0000", 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4498000000000001e-05 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338165+0000", 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401880+0000", 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063715331 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401978+0000", 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8158000000000004e-05 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402083+0000", 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104799 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55229 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.42487963, 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064053115999999993, 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55229 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338426+0000", 2024-09-16T22:17:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017367500000000001 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338431+0000", 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338442+0000", 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1749999999999999e-05 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338449+0000", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6649999999999997e-06 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402117+0000", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063668279999999994 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402207+0000", 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9257000000000003e-05 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402305+0000", 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8911000000000002e-05 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55231 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4968448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.424301440000001, 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911281, 2024-09-16T22:17:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55231 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-07 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338834+0000", 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-16T22:17:12.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338841+0000", 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338847+0000", 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1360000000000002e-06 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402566+0000", 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063718876999999993 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402647+0000", 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1384000000000005e-05 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402742+0000", 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4686999999999997e-05 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55232 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4399104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.424276438, 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064122381000000006, 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55232 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339005+0000", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014984300000000001 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339010+0000", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999999e-06 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339023+0000", 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2904e-05 2024-09-16T22:17:12.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339030+0000", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402777+0000", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063746612999999994 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402864+0000", 2024-09-16T22:17:12.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7818000000000005e-05 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402978+0000", 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113507 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55233 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4984832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:12.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.424090491999999, 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151505999999997, 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55233 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339210+0000", 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016822699999999999 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.741e-06 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339225+0000", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0149000000000001e-05 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339232+0000", 2024-09-16T22:17:12.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403011+0000", 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063779340000000004 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403096+0000", 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5482000000000006e-05 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403193+0000", 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6496999999999994e-05 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55234 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.423884128999999, 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151969000000003, 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55234 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339412+0000", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000164544 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339417+0000", 2024-09-16T22:17:12.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6160000000000004e-06 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339428+0000", 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0522e-05 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339435+0000", 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7880000000000007e-06 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403227+0000", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063791688999999999 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403309+0000", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2254000000000002e-05 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403400+0000", 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0556000000000005e-05 2024-09-16T22:17:12.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55235 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5001216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.423684536, 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064168931999999998, 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55235 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339684+0000", 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023680199999999999 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339690+0000", 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.049e-06 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339701+0000", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0494e-05 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339706+0000", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6579999999999999e-06 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403431+0000", 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063724905999999998 2024-09-16T22:17:12.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403519+0000", 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7892999999999993e-05 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403616+0000", 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.713e-05 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55236 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4431872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.423414879999999, 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064115839999999993, 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55236 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339965+0000", 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024771199999999998 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339970+0000", 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339980+0000", 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0254999999999999e-05 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339987+0000", 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7519999999999999e-06 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403650+0000", 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063663553999999997 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403736+0000", 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5778000000000004e-05 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403833+0000", 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6854000000000005e-05 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55237 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5017600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.423132919, 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064058452000000002, 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55237 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340000+0000", 2024-09-16T22:17:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-16T22:17:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340002+0000", 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5859999999999999e-06 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340009+0000", 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2080000000000003e-06 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340015+0000", 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403865+0000", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063850812000000007 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403955+0000", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9648999999999995e-05 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404057+0000", 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102472 2024-09-16T22:17:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55238 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4448256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.423108341000001, 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064252408999999996, 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55238 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340290+0000", 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026611899999999999 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340294+0000", 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2810000000000002e-06 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340302+0000", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4630000000000001e-06 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340308+0000", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8259999999999998e-06 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404092+0000", 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063783230999999996 2024-09-16T22:17:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404178+0000", 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6080000000000003e-05 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404276+0000", 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8408999999999999e-05 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55239 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5033984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.422811790000001, 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064172565000000001, 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55239 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341020+0000", 2024-09-16T22:17:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070000599999999998 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341025+0000", 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6589999999999998e-06 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341038+0000", 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3534e-05 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341046+0000", 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6179999999999997e-06 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404308+0000", 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063261957999999993 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404393+0000", 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4791999999999997e-05 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404493+0000", 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9998e-05 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55243 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5066752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.421948326000001, 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064032600999999995, 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55243 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341467+0000", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028289899999999998 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341471+0000", 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5569999999999997e-06 2024-09-16T22:17:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341481+0000", 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0339e-05 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341488+0000", 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405047+0000", 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559063999999998 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405129+0000", 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1919e-05 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405216+0000", 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7230000000000003e-05 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55244 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4497408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.421632639, 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063903336000000005, 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55244 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341673+0000", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017377700000000001 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341678+0000", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341688+0000", 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0200000000000001e-05 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341695+0000", 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.55e-06 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405249+0000", 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063554352999999994 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405323+0000", 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3677000000000006e-05 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405403+0000", 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9628000000000004e-05 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55245 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5083136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.421423535000001, 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063879072999999995, 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55245 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341930+0000", 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022177199999999999 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341940+0000", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341950+0000", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0124e-05 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341959+0000", 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7260000000000008e-06 2024-09-16T22:17:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405436+0000", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063477512 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405509+0000", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2156999999999994e-05 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405587+0000", 2024-09-16T22:17:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8957999999999999e-05 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:12.858 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:12.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:12.860 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:12.861+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:12.960 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:12.962+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:13.007 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:13.007 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:13.061 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:13.063+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:13.122 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:13.123 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:13.162 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:13.164+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:13.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:17:13.245 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.669203518, 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.669184177999998, 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.668779778000001, 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45048 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.667106333, 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676787000000002, 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45048 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579108+0000", 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579110+0000", 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579116+0000", 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579120+0000", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630608+0000", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051487796000000002 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630687+0000", 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8709999999999997e-05 2024-09-16T22:17:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630784+0000", 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7190999999999998e-05 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.667084507999999, 2024-09-16T22:17:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.667053975000002, 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.662653076000002, 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.662632595000002, 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.662612019000001, 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.662587534, 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.662565962999999, 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45147 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656508189, 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051674649000000003, 2024-09-16T22:17:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45147 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589706+0000", 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589708+0000", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9599999999999999e-06 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589714+0000", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589720+0000", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.383e-06 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641214+0000", 2024-09-16T22:17:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051494686999999997 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641285+0000", 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0795000000000002e-05 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641380+0000", 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5118999999999995e-05 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656487365, 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656467032999998, 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656445814000001, 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656424441999999, 2024-09-16T22:17:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656403604000001, 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656383023, 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.656361455999999, 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.651714582, 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.337 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:13.339 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone32 from 0x13813e7 (overlap 0x322f01) to 0x504b3 2024-09-16T22:17:13.339 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:13.340+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:13.413 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T22:17:13.439 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:13.441+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:13.518 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:13.519 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:13.540 INFO:teuthology.orchestra.run.smithi031.stdout:3428 write 0x6bee748 thru 0x6bf4c5c (0x6515 bytes) 2024-09-16T22:17:13.541 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:13.542+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:13.552 INFO:teuthology.orchestra.run.smithi031.stdout:3430 read 0x5ae4ecd thru 0x5aecffa (0x812e bytes) 2024-09-16T22:17:13.552 INFO:teuthology.orchestra.run.smithi031.stdout:3432 read 0x38cb5c3 thru 0x38d2ffc (0x7a3a bytes) 2024-09-16T22:17:13.567 INFO:teuthology.orchestra.run.smithi031.stdout:3433 write 0xddfc8a4 thru 0xde04643 (0x7da0 bytes) 2024-09-16T22:17:13.580 INFO:teuthology.orchestra.run.smithi031.stdout:3434 punch from 0x630a50b to 0x6315d5a, (0xb84f bytes) 2024-09-16T22:17:13.580 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.580 INFO:teuthology.orchestra.run.smithi031.stdout:3436 read 0xd81ad9d thru 0xd829ff9 (0xf25d bytes) 2024-09-16T22:17:13.581 INFO:teuthology.orchestra.run.smithi031.stdout:3437 read 0x45551fa thru 0x4556ff8 (0x1dff bytes) 2024-09-16T22:17:13.581 INFO:teuthology.orchestra.run.smithi031.stdout:3438 writesame 0x7642e8a thru 0x7644ef7 data_size 0x7(0x206e bytes) 2024-09-16T22:17:13.585 INFO:teuthology.orchestra.run.smithi031.stdout:3440 writesame 0x5bd193c thru 0x5bd8373 data_size 0x67(0x6a38 bytes) 2024-09-16T22:17:13.600 INFO:teuthology.orchestra.run.smithi031.stdout:3441 write 0xae7e10f thru 0xae8488a (0x677c bytes) 2024-09-16T22:17:13.613 INFO:teuthology.orchestra.run.smithi031.stdout:3442 read 0x7ca9675 thru 0x7caa2cc (0xc58 bytes) 2024-09-16T22:17:13.613 INFO:teuthology.orchestra.run.smithi031.stdout:3443 write 0xc30257e thru 0xc30cfe6 (0xaa69 bytes) 2024-09-16T22:17:13.634 INFO:teuthology.orchestra.run.smithi031.stdout:3444 write 0xdef463d thru 0xdefbd33 (0x76f7 bytes) 2024-09-16T22:17:13.641 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:13.643+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:13.646 INFO:teuthology.orchestra.run.smithi031.stdout:3445 read 0x6e80447 thru 0x6e8757e (0x7138 bytes) 2024-09-16T22:17:13.647 INFO:teuthology.orchestra.run.smithi031.stdout:3448 read 0xb12fa55 thru 0xb136360 (0x690c bytes) 2024-09-16T22:17:13.647 INFO:teuthology.orchestra.run.smithi031.stdout:3450 writesame 0x54914e thru 0x558c08 data_size 0x1(0xfabb bytes) 2024-09-16T22:17:13.685 INFO:teuthology.orchestra.run.smithi031.stdout:3451 writesame 0x60c263b thru 0x60cd90a data_size 0x8(0xb2d0 bytes) 2024-09-16T22:17:13.691 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:13.691 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.700 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.700 INFO:teuthology.orchestra.run.smithi031.stdout:3455 read 0x7d7cce thru 0x7e651f (0xe852 bytes) 2024-09-16T22:17:13.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:13.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.704 INFO:teuthology.orchestra.run.smithi031.stdout:3462 trunc from 0xdefbd34 to 0xcf4556b 2024-09-16T22:17:13.716 INFO:teuthology.orchestra.run.smithi031.stdout:3463 writesame 0xc248d37 thru 0xc2505c9 data_size 0x1(0x7893 bytes) 2024-09-16T22:17:13.731 INFO:teuthology.orchestra.run.smithi031.stdout:3464 punch from 0x551893d to 0x55217e1, (0x8ea4 bytes) 2024-09-16T22:17:13.732 INFO:teuthology.orchestra.run.smithi031.stdout:3465 read 0xa2d4c8f thru 0xa2da5fa (0x596c bytes) 2024-09-16T22:17:13.732 INFO:teuthology.orchestra.run.smithi031.stdout:3467 read 0x5aab251 thru 0x5abaf28 (0xfcd8 bytes) 2024-09-16T22:17:13.732 INFO:teuthology.orchestra.run.smithi031.stdout:3468 read 0xcde0904 thru 0xcdec85a (0xbf57 bytes) 2024-09-16T22:17:13.733 INFO:teuthology.orchestra.run.smithi031.stdout:3470 trunc from 0xcf4556b to 0xd91aacd 2024-09-16T22:17:13.738 INFO:teuthology.orchestra.run.smithi031.stdout:3471 write 0x92a0111 thru 0x92aa11f (0xa00f bytes) 2024-09-16T22:17:13.741 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:13.743+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:13.753 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.753 INFO:teuthology.orchestra.run.smithi031.stdout:3473 read 0x3adf250 thru 0x3aed886 (0xe637 bytes) 2024-09-16T22:17:13.753 INFO:teuthology.orchestra.run.smithi031.stdout:3474 write 0x701516a thru 0x7019d78 (0x4c0f bytes) 2024-09-16T22:17:13.760 INFO:teuthology.orchestra.run.smithi031.stdout:3476 write 0x556a63a thru 0x557489b (0xa262 bytes) 2024-09-16T22:17:13.775 INFO:teuthology.orchestra.run.smithi031.stdout:3477 read 0x69f1c19 thru 0x69fef6d (0xd355 bytes) 2024-09-16T22:17:13.781 INFO:teuthology.orchestra.run.smithi031.stdout:3478 write 0xe4c119d thru 0xe4c2304 (0x1168 bytes) 2024-09-16T22:17:13.785 INFO:teuthology.orchestra.run.smithi031.stdout:3479 punch from 0x15c7211 to 0x15d0e4e, (0x9c3d bytes) 2024-09-16T22:17:13.785 INFO:teuthology.orchestra.run.smithi031.stdout:3480 write 0x9954130 thru 0x9955878 (0x1749 bytes) 2024-09-16T22:17:13.790 INFO:teuthology.orchestra.run.smithi031.stdout:3481 read 0x8c09c06 thru 0x8c1533a (0xb735 bytes) 2024-09-16T22:17:13.790 INFO:teuthology.orchestra.run.smithi031.stdout:3482 punch from 0x73a0cd7 to 0x73ae648, (0xd971 bytes) 2024-09-16T22:17:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:3483 writesame 0x3f42544 thru 0x3f4afb3 data_size 0x8(0x8a70 bytes) 2024-09-16T22:17:13.802 INFO:teuthology.orchestra.run.smithi031.stdout:3484 read 0x28a1e37 thru 0x28aa2b8 (0x8482 bytes) 2024-09-16T22:17:13.808 INFO:teuthology.orchestra.run.smithi031.stdout:3486 write 0xead27b5 thru 0xeadce10 (0xa65c bytes) 2024-09-16T22:17:13.838 INFO:teuthology.orchestra.run.smithi031.stdout:3487 trunc from 0xeadce11 to 0x484ee83 2024-09-16T22:17:13.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.414755174000007, 2024-09-16T22:17:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.579950122, 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.579929283, 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.579906042999999, 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.579204158, 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.576071711000001, 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.575662101000001, 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.575375923999999, 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.575078700000001, 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.574290457, 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.574175688, 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.573924471, 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.573616113, 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.573577176000001, 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.573291756, 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567440253000001, 2024-09-16T22:17:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567415855, 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567365205, 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567331508000001, 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567220538000001, 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:13.935 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:13.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3489 read 0x40f58e2 thru 0x40fea87 (0x91a6 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3490 read 0x2db5135 thru 0x2dba261 (0x512d bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3491 read 0x1b9077c thru 0x1b9edbc (0xe641 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3492 punch from 0x11c2baf to 0x11c6871, (0x3cc2 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3493 write 0x82e1c92 thru 0x82ef442 (0xd7b1 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3494 write 0x62da466 thru 0x62df41f (0x4fba bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3495 writesame 0x81e20f1 thru 0x81ee949 data_size 0x11(0xc859 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:3496 write 0x9143c3f thru 0x914de33 (0xa1f5 bytes) 2024-09-16T22:17:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:13.938 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:13.938+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:13.939 INFO:teuthology.orchestra.run.smithi031.stdout:3498 trunc from 0x914de34 to 0x44ae274 2024-09-16T22:17:13.954 INFO:teuthology.orchestra.run.smithi031.stdout:3501 write 0x143a4f4 thru 0x144678a (0xc297 bytes) 2024-09-16T22:17:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:3503 write 0xc4d63d9 thru 0xc4e2ce5 (0xc90d bytes) 2024-09-16T22:17:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:3506 read 0x8bd351 thru 0x8c29a0 (0x5650 bytes) 2024-09-16T22:17:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:3507 read 0xb8ec12e thru 0xb8f9283 (0xd156 bytes) 2024-09-16T22:17:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:3508 punch from 0x6dc3fcd to 0x6dcbdfb, (0x7e2e bytes) 2024-09-16T22:17:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:3509 read 0x1091137 thru 0x10933a6 (0x2270 bytes) 2024-09-16T22:17:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:3510 read 0x63ffe68 thru 0x6401d99 (0x1f32 bytes) 2024-09-16T22:17:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:3511 write 0xb30d54f thru 0xb316943 (0x93f5 bytes) 2024-09-16T22:17:14.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:3512 punch from 0x944db10 to 0x944fa29, (0x1f19 bytes) 2024-09-16T22:17:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:3516 writesame 0x40b2849 thru 0x40b6703 data_size 0x35(0x3ebb bytes) 2024-09-16T22:17:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:3517 write 0x942b046 thru 0x94320e3 (0x709e bytes) 2024-09-16T22:17:14.036 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:14.038+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:14.052 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:14.052 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:14.052 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:14.052 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:3519 write 0xed26af9 thru 0xed2a57a (0x3a82 bytes) 2024-09-16T22:17:14.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:3520 write 0x8fadf56 thru 0x8fba9de (0xca89 bytes) 2024-09-16T22:17:14.085 INFO:teuthology.orchestra.run.smithi031.stdout:3522 trunc from 0xed2a57b to 0xe329f58 2024-09-16T22:17:14.095 INFO:teuthology.orchestra.run.smithi031.stdout:3524 write 0x2083986 thru 0x2086d56 (0x33d1 bytes) 2024-09-16T22:17:14.104 INFO:teuthology.orchestra.run.smithi031.stdout:3526 write 0x9da1cc9 thru 0x9db1c1d (0xff55 bytes) 2024-09-16T22:17:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:3531 read 0x4455e42 thru 0x4458e7b (0x303a bytes) 2024-09-16T22:17:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:3533 trunc from 0xe329f58 to 0x5786872 2024-09-16T22:17:14.138 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:14.139+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:3534 read 0x879824 thru 0x8827bc (0x8f99 bytes) 2024-09-16T22:17:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:3535 read 0x22ec4bd thru 0x22f8bad (0xc6f1 bytes) 2024-09-16T22:17:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:3538 writesame 0xc77ddc9 thru 0xc783126 data_size 0x3(0x535e bytes) 2024-09-16T22:17:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:3540 read 0xdf2053 thru 0xdfcb94 (0xab42 bytes) 2024-09-16T22:17:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:3541 clone 34 order 22 su 131072 sc 10 2024-09-16T22:17:14.177 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:14.177 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:14.194 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:14.210 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:14.216 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:17:14.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:14.232 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:14.250 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:14.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:14.263 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:14.265+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:14.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:14.268 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:14.308 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:14.309 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone33 intact 2024-09-16T22:17:14.327 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:14.327 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:14.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:14.377 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:14.379+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:14.440 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:14.442+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:14.482 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:14.482 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:14.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:14.494 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T22:17:14.508 INFO:teuthology.orchestra.run.smithi031.stdout:3542 trunc from 0xc783127 to 0x3ad7e68 2024-09-16T22:17:14.553 INFO:teuthology.orchestra.run.smithi031.stdout:3544 writesame 0xb08141e thru 0xb088ddb data_size 0x1(0x79be bytes) 2024-09-16T22:17:14.561 INFO:teuthology.orchestra.run.smithi031.stdout:3545 writesame 0xd0d362d thru 0xd0d5e11 data_size 0x1(0x27e5 bytes) 2024-09-16T22:17:14.561 INFO:teuthology.orchestra.run.smithi031.stdout:3546 punch from 0x5b8a2e4 to 0x5b97d10, (0xda2c bytes) 2024-09-16T22:17:14.561 INFO:teuthology.orchestra.run.smithi031.stdout:3547 trunc from 0xd0d5e12 to 0xd191970 2024-09-16T22:17:14.565 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.565 INFO:teuthology.orchestra.run.smithi031.stdout:3549 punch from 0x79e75a9 to 0x79f44f9, (0xcf50 bytes) 2024-09-16T22:17:14.566 INFO:teuthology.orchestra.run.smithi031.stdout:3550 writesame 0x88756f9 thru 0x8881062 data_size 0x51(0xb96a bytes) 2024-09-16T22:17:14.569 INFO:teuthology.orchestra.run.smithi031.stdout:3551 read 0x7f376d6 thru 0x7f3be1e (0x4749 bytes) 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:3555 read 0xbdd22be thru 0xbddb977 (0x96ba bytes) 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:3557 punch from 0x48428b7 to 0x484d045, (0xa78e bytes) 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:3560 punch from 0x4b3596a to 0x4b42952, (0xcfe8 bytes) 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:3561 read 0x69274b5 thru 0x692f9ee (0x853a bytes) 2024-09-16T22:17:14.570 INFO:teuthology.orchestra.run.smithi031.stdout:3562 write 0x3ab8d5e thru 0x3abd851 (0x4af4 bytes) 2024-09-16T22:17:14.571 INFO:teuthology.orchestra.run.smithi031.stdout:3563 read 0x7f87ecd thru 0x7f8c0a0 (0x41d4 bytes) 2024-09-16T22:17:14.571 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.571 INFO:teuthology.orchestra.run.smithi031.stdout:3565 write 0xc5989c9 thru 0xc59bbb9 (0x31f1 bytes) 2024-09-16T22:17:14.574 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:14.576+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:14.581 INFO:teuthology.orchestra.run.smithi031.stdout:3566 trunc from 0xd191970 to 0x6d4dbd7 2024-09-16T22:17:14.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.590 INFO:teuthology.orchestra.run.smithi031.stdout:3568 read 0x153c11 thru 0x15e39c (0xa78c bytes) 2024-09-16T22:17:14.593 INFO:teuthology.orchestra.run.smithi031.stdout:3570 write 0x2d5795e thru 0x2d612f3 (0x9996 bytes) 2024-09-16T22:17:14.604 INFO:teuthology.orchestra.run.smithi031.stdout:3571 writesame 0xa9b57ba thru 0xa9c18db data_size 0x3b(0xc122 bytes) 2024-09-16T22:17:14.604 INFO:teuthology.orchestra.run.smithi031.stdout:3572 write 0x6a78ab0 thru 0x6a82662 (0x9bb3 bytes) 2024-09-16T22:17:14.608 INFO:teuthology.orchestra.run.smithi031.stdout:3573 punch from 0xa797a7e to 0xa79e347, (0x68c9 bytes) 2024-09-16T22:17:14.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.609 INFO:teuthology.orchestra.run.smithi031.stdout:3577 punch from 0x930ee40 to 0x9314904, (0x5ac4 bytes) 2024-09-16T22:17:14.613 INFO:teuthology.orchestra.run.smithi031.stdout:3578 trunc from 0xa9c18dc to 0xeaa6c71 2024-09-16T22:17:14.618 INFO:teuthology.orchestra.run.smithi031.stdout:3579 read 0x624fa16 thru 0x625584c (0x5e37 bytes) 2024-09-16T22:17:14.618 INFO:teuthology.orchestra.run.smithi031.stdout:3580 write 0xb3e5be4 thru 0xb3f2272 (0xc68f bytes) 2024-09-16T22:17:14.621 INFO:teuthology.orchestra.run.smithi031.stdout:3581 writesame 0x44984aa thru 0x44a405a data_size 0x1(0xbbb1 bytes) 2024-09-16T22:17:14.638 INFO:teuthology.orchestra.run.smithi031.stdout:3582 read 0x25b3bb4 thru 0x25c19d1 (0xde1e bytes) 2024-09-16T22:17:14.638 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.638 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:14.639 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:14.641 INFO:teuthology.orchestra.run.smithi031.stdout:3585 trunc from 0xeaa6c71 to 0xbf77ff7 2024-09-16T22:17:14.641 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:14.643+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:14.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:14.648 INFO:teuthology.orchestra.run.smithi031.stdout:3586 write 0x77974fd thru 0x779e7a6 (0x72aa bytes) 2024-09-16T22:17:14.651 INFO:teuthology.orchestra.run.smithi031.stdout:3587 write 0x292e4d5 thru 0x2934d18 (0x6844 bytes) 2024-09-16T22:17:14.653 INFO:teuthology.orchestra.run.smithi031.stdout:3588 writesame 0x848261a thru 0x84895aa data_size 0xd(0x6f91 bytes) 2024-09-16T22:17:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:3590 read 0x24667fc thru 0x2469645 (0x2e4a bytes) 2024-09-16T22:17:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:3591 read 0x29a6967 thru 0x29b1280 (0xa91a bytes) 2024-09-16T22:17:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:3594 read 0x37f3355 thru 0x37f33f6 (0xa2 bytes) 2024-09-16T22:17:14.657 INFO:teuthology.orchestra.run.smithi031.stdout:3595 write 0x35e52af thru 0x35e97c6 (0x4518 bytes) 2024-09-16T22:17:14.667 INFO:teuthology.orchestra.run.smithi031.stdout:3596 writesame 0xe85e112 thru 0xe868c51 data_size 0x28(0xab40 bytes) 2024-09-16T22:17:14.667 INFO:teuthology.orchestra.run.smithi031.stdout:3597 write 0x2218786 thru 0x221d6a9 (0x4f24 bytes) 2024-09-16T22:17:14.696 INFO:teuthology.orchestra.run.smithi031.stdout:3598 writesame 0xd47f88f thru 0xd47fe28 data_size 0x3(0x59a bytes) 2024-09-16T22:17:14.701 INFO:teuthology.orchestra.run.smithi031.stdout:3599 trunc from 0xe868c52 to 0xca383fd 2024-09-16T22:17:14.708 INFO:teuthology.orchestra.run.smithi031.stdout:3600 read 0x374b249 thru 0x374bcb7 (0xa6f bytes) 2024-09-16T22:17:14.708 INFO:teuthology.orchestra.run.smithi031.stdout:3601 read 0xa547a13 thru 0xa5493e2 (0x19d0 bytes) 2024-09-16T22:17:14.708 INFO:teuthology.orchestra.run.smithi031.stdout:3602 write 0xbbad832 thru 0xbbaff0e (0x26dd bytes) 2024-09-16T22:17:14.711 INFO:teuthology.orchestra.run.smithi031.stdout:3603 writesame 0x15c09fb thru 0x15ca7c2 data_size 0x99(0x9dc8 bytes) 2024-09-16T22:17:14.737 INFO:teuthology.orchestra.run.smithi031.stdout:3604 writesame 0xcb14205 thru 0xcb19dcd data_size 0x1(0x5bc9 bytes) 2024-09-16T22:17:14.739 INFO:teuthology.orchestra.run.smithi031.stdout:3605 trunc from 0xcb19dce to 0xa72c521 2024-09-16T22:17:14.742 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:14.744+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.750 INFO:teuthology.orchestra.run.smithi031.stdout:3607 write 0x69384c5 thru 0x6941be7 (0x9723 bytes) 2024-09-16T22:17:14.751 INFO:teuthology.orchestra.run.smithi031.stdout:3608 punch from 0x688fc6 to 0x68926c, (0x2a6 bytes) 2024-09-16T22:17:14.755 INFO:teuthology.orchestra.run.smithi031.stdout:3610 trunc from 0xa72c521 to 0xd6fb6c4 2024-09-16T22:17:14.761 INFO:teuthology.orchestra.run.smithi031.stdout:3611 write 0x41a5b2 thru 0x4294e2 (0xef31 bytes) 2024-09-16T22:17:14.784 INFO:teuthology.orchestra.run.smithi031.stdout:3612 read 0xc56dc4c thru 0xc57d37f (0xf734 bytes) 2024-09-16T22:17:14.784 INFO:teuthology.orchestra.run.smithi031.stdout:3614 read 0x505d1d3 thru 0x5064c7d (0x7aab bytes) 2024-09-16T22:17:14.790 INFO:teuthology.orchestra.run.smithi031.stdout:3615 trunc from 0xd6fb6c4 to 0x8bdf2b4 2024-09-16T22:17:14.804 INFO:teuthology.orchestra.run.smithi031.stdout:3616 trunc from 0x8bdf2b4 to 0xb50ca4c 2024-09-16T22:17:14.804 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:14.804 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:14.809 INFO:teuthology.orchestra.run.smithi031.stdout:3617 read 0x7a35204 thru 0x7a3e02a (0x8e27 bytes) 2024-09-16T22:17:14.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:14.814 INFO:teuthology.orchestra.run.smithi031.stdout:3618 write 0xb8b2132 thru 0xb8b28ea (0x7b9 bytes) 2024-09-16T22:17:14.817 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.817 INFO:teuthology.orchestra.run.smithi031.stdout:3620 write 0x95cc4c1 thru 0x95d91e4 (0xcd24 bytes) 2024-09-16T22:17:14.820 INFO:teuthology.orchestra.run.smithi031.stdout:3621 read 0x592bf63 thru 0x593ae15 (0xeeb3 bytes) 2024-09-16T22:17:14.820 INFO:teuthology.orchestra.run.smithi031.stdout:3622 writesame 0x4a21a61 thru 0x4a2832f data_size 0x1(0x68cf bytes) 2024-09-16T22:17:14.833 INFO:teuthology.orchestra.run.smithi031.stdout:3624 read 0x94e2bd thru 0x954f3c (0x6c80 bytes) 2024-09-16T22:17:14.833 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:14.833 INFO:teuthology.orchestra.run.smithi031.stdout:3626 read 0x4a066ab thru 0x4a10cdf (0xa635 bytes) 2024-09-16T22:17:14.834 INFO:teuthology.orchestra.run.smithi031.stdout:3627 punch from 0x5fcd227 to 0x5fdc48b, (0xf264 bytes) 2024-09-16T22:17:14.843 INFO:teuthology.orchestra.run.smithi031.stdout:3628 writesame 0xedc1045 thru 0xedc92f7 data_size 0x3(0x82b3 bytes) 2024-09-16T22:17:14.843 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:14.845+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:14.844 INFO:teuthology.orchestra.run.smithi031.stdout:3630 trunc from 0xedc92f8 to 0xe0af9b8 2024-09-16T22:17:14.849 INFO:teuthology.orchestra.run.smithi031.stdout:3631 read 0x8fab0fc thru 0x8fabbf7 (0xafc bytes) 2024-09-16T22:17:14.849 INFO:teuthology.orchestra.run.smithi031.stdout:3632 write 0xc87205e thru 0xc880a4a (0xe9ed bytes) 2024-09-16T22:17:14.853 INFO:teuthology.orchestra.run.smithi031.stdout:3633 read 0x61e3604 thru 0x61e4c2b (0x1628 bytes) 2024-09-16T22:17:14.853 INFO:teuthology.orchestra.run.smithi031.stdout:3635 read 0x727663 thru 0x72b727 (0x40c5 bytes) 2024-09-16T22:17:14.854 INFO:teuthology.orchestra.run.smithi031.stdout:3636 writesame 0x1fc1b85 thru 0x1fcaa35 data_size 0x1(0x8eb1 bytes) 2024-09-16T22:17:14.872 INFO:teuthology.orchestra.run.smithi031.stdout:3637 writesame 0x207cb9e thru 0x2080710 data_size 0x39(0x3b73 bytes) 2024-09-16T22:17:14.922 INFO:teuthology.orchestra.run.smithi031.stdout:3638 punch from 0xb160c2d to 0xb163533, (0x2906 bytes) 2024-09-16T22:17:14.922 INFO:teuthology.orchestra.run.smithi031.stdout:3639 write 0x5ba23f3 thru 0x5ba3c29 (0x1837 bytes) 2024-09-16T22:17:14.924 INFO:teuthology.orchestra.run.smithi031.stdout:3641 write 0xdc21d76 thru 0xdc30b80 (0xee0b bytes) 2024-09-16T22:17:14.928 INFO:teuthology.orchestra.run.smithi031.stdout:3643 writesame 0xd87cc46 thru 0xd882c08 data_size 0x1(0x5fc3 bytes) 2024-09-16T22:17:14.930 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:14.930 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:14.936 INFO:teuthology.orchestra.run.smithi031.stdout:3644 read 0x2fbe3fc thru 0x2fc55be (0x71c3 bytes) 2024-09-16T22:17:14.937 INFO:teuthology.orchestra.run.smithi031.stdout:3645 write 0xa6f0d80 thru 0xa6fd296 (0xc517 bytes) 2024-09-16T22:17:14.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:14.941 INFO:teuthology.orchestra.run.smithi031.stdout:3647 read 0xd99342b thru 0xd99ac2a (0x7800 bytes) 2024-09-16T22:17:14.941 INFO:teuthology.orchestra.run.smithi031.stdout:3648 write 0x63437f thru 0x63939b (0x501d bytes) 2024-09-16T22:17:14.943 INFO:teuthology.orchestra.run.smithi031.stdout:3649 punch from 0xb56677 to 0xb5cbc4, (0x654d bytes) 2024-09-16T22:17:14.943 INFO:teuthology.orchestra.run.smithi031.stdout:3650 punch from 0xe02f5f7 to 0xe0311b9, (0x1bc2 bytes) 2024-09-16T22:17:14.943 INFO:teuthology.orchestra.run.smithi031.stdout:3651 writesame 0x7e4757d thru 0x7e4bcc2 data_size 0x3(0x4746 bytes) 2024-09-16T22:17:14.947 INFO:teuthology.orchestra.run.smithi031.stdout:3652 punch from 0xac5f49f to 0xac6bbb5, (0xc716 bytes) 2024-09-16T22:17:14.947 INFO:teuthology.orchestra.run.smithi031.stdout:3653 write 0x9ff1fc7 thru 0x9ff3a8b (0x1ac5 bytes) 2024-09-16T22:17:14.950 INFO:teuthology.orchestra.run.smithi031.stdout:3656 writesame 0xa1f2c8d thru 0xa202a03 data_size 0x2b(0xfd77 bytes) 2024-09-16T22:17:14.954 INFO:teuthology.orchestra.run.smithi031.stdout:3657 read 0x3a0acce thru 0x3a12849 (0x7b7c bytes) 2024-09-16T22:17:14.955 INFO:teuthology.orchestra.run.smithi031.stdout:3658 write 0x92d5fe thru 0x936f08 (0x990b bytes) 2024-09-16T22:17:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:3659 write 0xdfa6708 thru 0xdfad26a (0x6b63 bytes) 2024-09-16T22:17:14.960 INFO:teuthology.orchestra.run.smithi031.stdout:3660 write 0x7af6ff thru 0x7bf236 (0xfb38 bytes) 2024-09-16T22:17:14.981 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:14.983+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:14.991 INFO:teuthology.orchestra.run.smithi031.stdout:3661 writesame 0x759f7d2 thru 0x75a4a0e data_size 0x1(0x523d bytes) 2024-09-16T22:17:15.001 INFO:teuthology.orchestra.run.smithi031.stdout:3663 write 0x9b34f39 thru 0x9b39161 (0x4229 bytes) 2024-09-16T22:17:15.004 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.004 INFO:teuthology.orchestra.run.smithi031.stdout:3666 write 0x80f2b7b thru 0x80f6665 (0x3aeb bytes) 2024-09-16T22:17:15.006 INFO:teuthology.orchestra.run.smithi031.stdout:3669 punch from 0x5fd2151 to 0x5fd2a75, (0x924 bytes) 2024-09-16T22:17:15.006 INFO:teuthology.orchestra.run.smithi031.stdout:3671 punch from 0x3dc065a to 0x3dcaf09, (0xa8af bytes) 2024-09-16T22:17:15.006 INFO:teuthology.orchestra.run.smithi031.stdout:3672 write 0x9ee1bc9 thru 0x9eee1d1 (0xc609 bytes) 2024-09-16T22:17:15.007 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.016 INFO:teuthology.orchestra.run.smithi031.stdout:3674 trunc from 0xe0af9b8 to 0x5dce923 2024-09-16T22:17:15.036 INFO:teuthology.orchestra.run.smithi031.stdout:3676 write 0x60ce915 thru 0x60d986b (0xaf57 bytes) 2024-09-16T22:17:15.039 INFO:teuthology.orchestra.run.smithi031.stdout:3677 punch from 0x650aa8 to 0x65891e, (0x7e76 bytes) 2024-09-16T22:17:15.039 INFO:teuthology.orchestra.run.smithi031.stdout:3678 read 0x24d776d thru 0x24e5476 (0xdd0a bytes) 2024-09-16T22:17:15.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.039 INFO:teuthology.orchestra.run.smithi031.stdout:3681 write 0x355349a thru 0x3559d67 (0x68ce bytes) 2024-09-16T22:17:15.040 INFO:teuthology.orchestra.run.smithi031.stdout:3682 punch from 0x4a6b2a6 to 0x4a6bcd9, (0xa33 bytes) 2024-09-16T22:17:15.044 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:15.046+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:15.046 INFO:teuthology.orchestra.run.smithi031.stdout:3683 write 0x6da830d thru 0x6da94b1 (0x11a5 bytes) 2024-09-16T22:17:15.049 INFO:teuthology.orchestra.run.smithi031.stdout:3684 punch from 0x6c9987e to 0x6ca58e9, (0xc06b bytes) 2024-09-16T22:17:15.053 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.053 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.058 INFO:teuthology.orchestra.run.smithi031.stdout:3685 write 0x9d0dcb4 thru 0x9d1d031 (0xf37e bytes) 2024-09-16T22:17:15.061 INFO:teuthology.orchestra.run.smithi031.stdout:3686 punch from 0x641812e to 0x641ae4e, (0x2d20 bytes) 2024-09-16T22:17:15.061 INFO:teuthology.orchestra.run.smithi031.stdout:3687 writesame 0x16d7c22 thru 0x16ddd17 data_size 0x15(0x60f6 bytes) 2024-09-16T22:17:15.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:3689 writesame 0x94efb4 thru 0x95cbed data_size 0x7(0xdc3a bytes) 2024-09-16T22:17:15.077 INFO:teuthology.orchestra.run.smithi031.stdout:3690 write 0xd0f9651 thru 0xd104fe6 (0xb996 bytes) 2024-09-16T22:17:15.081 INFO:teuthology.orchestra.run.smithi031.stdout:3691 read 0x94b00b7 thru 0x94b4156 (0x40a0 bytes) 2024-09-16T22:17:15.081 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.081 INFO:teuthology.orchestra.run.smithi031.stdout:3693 punch from 0x5aa0fdb to 0x5aa16d2, (0x6f7 bytes) 2024-09-16T22:17:15.081 INFO:teuthology.orchestra.run.smithi031.stdout:3694 punch from 0x796bb15 to 0x796cc76, (0x1161 bytes) 2024-09-16T22:17:15.087 INFO:teuthology.orchestra.run.smithi031.stdout:3697 write 0xe181a72 thru 0xe1865e0 (0x4b6f bytes) 2024-09-16T22:17:15.091 INFO:teuthology.orchestra.run.smithi031.stdout:3698 write 0x3b263b9 thru 0x3b34c95 (0xe8dd bytes) 2024-09-16T22:17:15.092 INFO:teuthology.orchestra.run.smithi031.stdout:3701 writesame 0xd634157 thru 0xd63dfe7 data_size 0x7(0x9e91 bytes) 2024-09-16T22:17:15.097 INFO:teuthology.orchestra.run.smithi031.stdout:3703 read 0xa986d42 thru 0xa989a3b (0x2cfa bytes) 2024-09-16T22:17:15.097 INFO:teuthology.orchestra.run.smithi031.stdout:3704 punch from 0x5da5c11 to 0x5da8b3b, (0x2f2a bytes) 2024-09-16T22:17:15.097 INFO:teuthology.orchestra.run.smithi031.stdout:3705 write 0x80d990d thru 0x80e6958 (0xd04c bytes) 2024-09-16T22:17:15.101 INFO:teuthology.orchestra.run.smithi031.stdout:3706 write 0x81613c0 thru 0x816a5e1 (0x9222 bytes) 2024-09-16T22:17:15.103 INFO:teuthology.orchestra.run.smithi031.stdout:3707 punch from 0xcf4b246 to 0xcf53284, (0x803e bytes) 2024-09-16T22:17:15.104 INFO:teuthology.orchestra.run.smithi031.stdout:3708 write 0x699248d thru 0x6999ca1 (0x7815 bytes) 2024-09-16T22:17:15.105 INFO:teuthology.orchestra.run.smithi031.stdout:3709 write 0x49bd69a thru 0x49c5727 (0x808e bytes) 2024-09-16T22:17:15.107 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.107 INFO:teuthology.orchestra.run.smithi031.stdout:3711 write 0xaf582a3 thru 0xaf5b060 (0x2dbe bytes) 2024-09-16T22:17:15.110 INFO:teuthology.orchestra.run.smithi031.stdout:3713 read 0x87595c9 thru 0x8766fa9 (0xd9e1 bytes) 2024-09-16T22:17:15.110 INFO:teuthology.orchestra.run.smithi031.stdout:3716 punch from 0x3a14113 to 0x3a22db2, (0xec9f bytes) 2024-09-16T22:17:15.110 INFO:teuthology.orchestra.run.smithi031.stdout:3717 read 0x4ab6388 thru 0x4ab6626 (0x29f bytes) 2024-09-16T22:17:15.110 INFO:teuthology.orchestra.run.smithi031.stdout:3718 read 0x5dd69ee thru 0x5de4cdf (0xe2f2 bytes) 2024-09-16T22:17:15.111 INFO:teuthology.orchestra.run.smithi031.stdout:3720 read 0xbe78c63 thru 0xbe8640e (0xd7ac bytes) 2024-09-16T22:17:15.111 INFO:teuthology.orchestra.run.smithi031.stdout:3721 write 0xcc857cd thru 0xcc94e13 (0xf647 bytes) 2024-09-16T22:17:15.113 INFO:teuthology.orchestra.run.smithi031.stdout:3722 read 0x41a3625 thru 0x41a873c (0x5118 bytes) 2024-09-16T22:17:15.114 INFO:teuthology.orchestra.run.smithi031.stdout:3723 writesame 0x94c7aa thru 0x959d75 data_size 0x6(0xd5cc bytes) 2024-09-16T22:17:15.125 INFO:teuthology.orchestra.run.smithi031.stdout:3724 trunc from 0xe1865e1 to 0x8f95b74 2024-09-16T22:17:15.137 INFO:teuthology.orchestra.run.smithi031.stdout:3725 read 0x114c121 thru 0x115beaf (0xfd8f bytes) 2024-09-16T22:17:15.140 INFO:teuthology.orchestra.run.smithi031.stdout:3727 read 0x49583f0 thru 0x4960660 (0x8271 bytes) 2024-09-16T22:17:15.140 INFO:teuthology.orchestra.run.smithi031.stdout:3729 write 0x790618d thru 0x790fe0c (0x9c80 bytes) 2024-09-16T22:17:15.145 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:15.147+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:15.147 INFO:teuthology.orchestra.run.smithi031.stdout:3731 trunc from 0x8f95b74 to 0x5624c73 2024-09-16T22:17:15.160 INFO:teuthology.orchestra.run.smithi031.stdout:3732 read 0x1e0c258 thru 0x1e0cc04 (0x9ad bytes) 2024-09-16T22:17:15.171 INFO:teuthology.orchestra.run.smithi031.stdout:3733 writesame 0xa42b2ef thru 0xa42ccc0 data_size 0x5(0x19d2 bytes) 2024-09-16T22:17:15.171 INFO:teuthology.orchestra.run.smithi031.stdout:3734 writesame 0x6993265 thru 0x699d291 data_size 0x5(0xa02d bytes) 2024-09-16T22:17:15.176 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.176 INFO:teuthology.orchestra.run.smithi031.stdout:3736 write 0x9754c2 thru 0x983707 (0xe246 bytes) 2024-09-16T22:17:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.222 INFO:teuthology.orchestra.run.smithi031.stdout:3737 write 0x85f9c7b thru 0x85ff44f (0x57d5 bytes) 2024-09-16T22:17:15.226 INFO:teuthology.orchestra.run.smithi031.stdout:3738 write 0x9d3dc6d thru 0x9d49347 (0xb6db bytes) 2024-09-16T22:17:15.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:15.233 INFO:teuthology.orchestra.run.smithi031.stdout:3739 writesame 0x2f3c115 thru 0x2f3c92a data_size 0x12(0x816 bytes) 2024-09-16T22:17:15.243 INFO:teuthology.orchestra.run.smithi031.stdout:3741 write 0xc5468f9 thru 0xc5560df (0xf7e7 bytes) 2024-09-16T22:17:15.246 INFO:teuthology.orchestra.run.smithi031.stdout:3742 punch from 0x732c407 to 0x73310ed, (0x4ce6 bytes) 2024-09-16T22:17:15.246 INFO:teuthology.orchestra.run.smithi031.stdout:3743 writesame 0x9110376 thru 0x9110f09 data_size 0x1a(0xb94 bytes) 2024-09-16T22:17:15.251 INFO:teuthology.orchestra.run.smithi031.stdout:3746 trunc from 0xc5560e0 to 0xe33000b 2024-09-16T22:17:15.256 INFO:teuthology.orchestra.run.smithi031.stdout:3747 write 0x95f3137 thru 0x95f645b (0x3325 bytes) 2024-09-16T22:17:15.265 INFO:teuthology.orchestra.run.smithi031.stdout:3748 writesame 0xea7c653 thru 0xea80f3c data_size 0x9(0x48ea bytes) 2024-09-16T22:17:15.265 INFO:teuthology.orchestra.run.smithi031.stdout:3749 punch from 0x2cd3d29 to 0x2ce19b9, (0xdc90 bytes) 2024-09-16T22:17:15.265 INFO:teuthology.orchestra.run.smithi031.stdout:3750 write 0xf0e4b9 thru 0xf1c087 (0xdbcf bytes) 2024-09-16T22:17:15.267 INFO:teuthology.orchestra.run.smithi031.stdout:3751 punch from 0xc9b083b to 0xc9beb26, (0xe2eb bytes) 2024-09-16T22:17:15.267 INFO:teuthology.orchestra.run.smithi031.stdout:3752 read 0x12f43f7 thru 0x12fb5b5 (0x71bf bytes) 2024-09-16T22:17:15.267 INFO:teuthology.orchestra.run.smithi031.stdout:3754 read 0xcb0892d thru 0xcb146c8 (0xbd9c bytes) 2024-09-16T22:17:15.268 INFO:teuthology.orchestra.run.smithi031.stdout:3755 writesame 0x672fea0 thru 0x6739436 data_size 0x17(0x9597 bytes) 2024-09-16T22:17:15.271 INFO:teuthology.orchestra.run.smithi031.stdout:3756 writesame 0x67fa58b thru 0x680818f data_size 0xf(0xdc05 bytes) 2024-09-16T22:17:15.272 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:15.274+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:15.274 INFO:teuthology.orchestra.run.smithi031.stdout:3757 read 0x491284d thru 0x491bee2 (0x9696 bytes) 2024-09-16T22:17:15.274 INFO:teuthology.orchestra.run.smithi031.stdout:3758 read 0x2f2286 thru 0x2f7f32 (0x5cad bytes) 2024-09-16T22:17:15.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.275 INFO:teuthology.orchestra.run.smithi031.stdout:3761 punch from 0xcb1b093 to 0xcb20f74, (0x5ee1 bytes) 2024-09-16T22:17:15.284 INFO:teuthology.orchestra.run.smithi031.stdout:3762 trunc from 0xea80f3d to 0x8624e51 2024-09-16T22:17:15.297 INFO:teuthology.orchestra.run.smithi031.stdout:3763 write 0x7dd82a8 thru 0x7de37fc (0xb555 bytes) 2024-09-16T22:17:15.301 INFO:teuthology.orchestra.run.smithi031.stdout:3765 read 0x65dcf09 thru 0x65e05a8 (0x36a0 bytes) 2024-09-16T22:17:15.302 INFO:teuthology.orchestra.run.smithi031.stdout:3766 write 0x2ea722c thru 0x2eaf54d (0x8322 bytes) 2024-09-16T22:17:15.311 INFO:teuthology.orchestra.run.smithi031.stdout:3767 trunc from 0x8624e51 to 0xe15b9e7 2024-09-16T22:17:15.316 INFO:teuthology.orchestra.run.smithi031.stdout:3768 writesame 0xdd32328 thru 0xdd36f20 data_size 0x3(0x4bf9 bytes) 2024-09-16T22:17:15.330 INFO:teuthology.orchestra.run.smithi031.stdout:3769 trunc from 0xe15b9e7 to 0x6e1f3ed 2024-09-16T22:17:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.347 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:15.348+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:3773 trunc from 0x6e1f3ed to 0x9f5e048 2024-09-16T22:17:15.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:3774 write 0x843adc thru 0x84db28 (0xa04d bytes) 2024-09-16T22:17:15.354 INFO:teuthology.orchestra.run.smithi031.stdout:3775 punch from 0x13bd78a to 0x13ca500, (0xcd76 bytes) 2024-09-16T22:17:15.364 INFO:teuthology.orchestra.run.smithi031.stdout:3776 trunc from 0x9f5e048 to 0xb717e2 2024-09-16T22:17:15.420 INFO:teuthology.orchestra.run.smithi031.stdout:3777 write 0x6932fbe thru 0x6933d56 (0xd99 bytes) 2024-09-16T22:17:15.423 INFO:teuthology.orchestra.run.smithi031.stdout:3779 writesame 0x174afa1 thru 0x1755b12 data_size 0x46(0xab72 bytes) 2024-09-16T22:17:15.424 INFO:teuthology.orchestra.run.smithi031.stdout:3780 read 0x2ab9ad0 thru 0x2abe106 (0x4637 bytes) 2024-09-16T22:17:15.438 INFO:teuthology.orchestra.run.smithi031.stdout:3784 writesame 0xd8e009b thru 0xd8eafc4 data_size 0x7(0xaf2a bytes) 2024-09-16T22:17:15.438 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.438 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.441 INFO:teuthology.orchestra.run.smithi031.stdout:3787 trunc from 0xd8eafc5 to 0xb06604c 2024-09-16T22:17:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:3790 punch from 0x470477 to 0x47330d, (0x2e96 bytes) 2024-09-16T22:17:15.447 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:15.449+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:15.448 INFO:teuthology.orchestra.run.smithi031.stdout:3791 trunc from 0xb06604c to 0xd316dcf 2024-09-16T22:17:15.454 INFO:teuthology.orchestra.run.smithi031.stdout:3792 punch from 0x3d4098c to 0x3d507c8, (0xfe3c bytes) 2024-09-16T22:17:15.454 INFO:teuthology.orchestra.run.smithi031.stdout:3793 read 0x4d7c097 thru 0x4d89437 (0xd3a1 bytes) 2024-09-16T22:17:15.454 INFO:teuthology.orchestra.run.smithi031.stdout:3794 write 0x42c26cb thru 0x42ce39b (0xbcd1 bytes) 2024-09-16T22:17:15.457 INFO:teuthology.orchestra.run.smithi031.stdout:3795 punch from 0x40f0cf8 to 0x40f302e, (0x2336 bytes) 2024-09-16T22:17:15.457 INFO:teuthology.orchestra.run.smithi031.stdout:3796 read 0x63220c3 thru 0x6328a64 (0x69a2 bytes) 2024-09-16T22:17:15.457 INFO:teuthology.orchestra.run.smithi031.stdout:3798 punch from 0xcffc25d to 0xd00be4d, (0xfbf0 bytes) 2024-09-16T22:17:15.465 INFO:teuthology.orchestra.run.smithi031.stdout:3799 trunc from 0xd316dcf to 0x6179bb7 2024-09-16T22:17:15.472 INFO:teuthology.orchestra.run.smithi031.stdout:3800 write 0x56c6bf7 thru 0x56d2908 (0xbd12 bytes) 2024-09-16T22:17:15.475 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.475 INFO:teuthology.orchestra.run.smithi031.stdout:3802 write 0x35f3fce thru 0x36030ec (0xf11f bytes) 2024-09-16T22:17:15.479 INFO:teuthology.orchestra.run.smithi031.stdout:3803 read 0x29197d9 thru 0x29225f4 (0x8e1c bytes) 2024-09-16T22:17:15.479 INFO:teuthology.orchestra.run.smithi031.stdout:3804 read 0x3f32c50 thru 0x3f3b6f5 (0x8aa6 bytes) 2024-09-16T22:17:15.479 INFO:teuthology.orchestra.run.smithi031.stdout:3807 write 0x451c081 thru 0x4527096 (0xb016 bytes) 2024-09-16T22:17:15.482 INFO:teuthology.orchestra.run.smithi031.stdout:3808 punch from 0x1feb49a to 0x1ff4632, (0x9198 bytes) 2024-09-16T22:17:15.491 INFO:teuthology.orchestra.run.smithi031.stdout:3809 writesame 0x93d3202 thru 0x93d7efc data_size 0x1(0x4cfb bytes) 2024-09-16T22:17:15.491 INFO:teuthology.orchestra.run.smithi031.stdout:3810 write 0x7c6cec thru 0x7c7667 (0x97c bytes) 2024-09-16T22:17:15.493 INFO:teuthology.orchestra.run.smithi031.stdout:3812 write 0x6a031d9 thru 0x6a09901 (0x6729 bytes) 2024-09-16T22:17:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.496 INFO:teuthology.orchestra.run.smithi031.stdout:3814 punch from 0x13c3a44 to 0x13c6101, (0x26bd bytes) 2024-09-16T22:17:15.496 INFO:teuthology.orchestra.run.smithi031.stdout:3815 read 0x8431966 thru 0x8431dfa (0x495 bytes) 2024-09-16T22:17:15.497 INFO:teuthology.orchestra.run.smithi031.stdout:3816 write 0x47bca45 thru 0x47c2a29 (0x5fe5 bytes) 2024-09-16T22:17:15.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:15.509 INFO:teuthology.orchestra.run.smithi031.stdout:3817 writesame 0xe0f095f thru 0xe0ffa96 data_size 0xba(0xf138 bytes) 2024-09-16T22:17:15.509 INFO:teuthology.orchestra.run.smithi031.stdout:3818 writesame 0x881a889 thru 0x8824aad data_size 0x1f(0xa225 bytes) 2024-09-16T22:17:15.512 INFO:teuthology.orchestra.run.smithi031.stdout:3820 write 0x10a2fc2 thru 0x10a5845 (0x2884 bytes) 2024-09-16T22:17:15.514 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.514 INFO:teuthology.orchestra.run.smithi031.stdout:3822 punch from 0xc1123d6 to 0xc1185da, (0x6204 bytes) 2024-09-16T22:17:15.514 INFO:teuthology.orchestra.run.smithi031.stdout:3823 write 0x5dd776b thru 0x5ddb4c0 (0x3d56 bytes) 2024-09-16T22:17:15.517 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.517 INFO:teuthology.orchestra.run.smithi031.stdout:3825 write 0xc48818e thru 0xc48ae12 (0x2c85 bytes) 2024-09-16T22:17:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:3826 read 0xc7a5247 thru 0xc7af5eb (0xa3a5 bytes) 2024-09-16T22:17:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:3827 read 0x4b8fadb thru 0x4b9ccab (0xd1d1 bytes) 2024-09-16T22:17:15.531 INFO:teuthology.orchestra.run.smithi031.stdout:3828 trunc from 0xe0ffa97 to 0x36daffb 2024-09-16T22:17:15.548 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:15.550+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:15.550 INFO:teuthology.orchestra.run.smithi031.stdout:3829 writesame 0x4f61ea9 thru 0x4f71230 data_size 0x4(0xf388 bytes) 2024-09-16T22:17:15.560 INFO:teuthology.orchestra.run.smithi031.stdout:3830 write 0x8ebb248 thru 0x8ec8922 (0xd6db bytes) 2024-09-16T22:17:15.576 INFO:teuthology.orchestra.run.smithi031.stdout:3831 writesame 0xde3e87c thru 0xde47e52 data_size 0x1(0x95d7 bytes) 2024-09-16T22:17:15.576 INFO:teuthology.orchestra.run.smithi031.stdout:3835 read 0xba8d8eb thru 0xba933a4 (0x5aba bytes) 2024-09-16T22:17:15.576 INFO:teuthology.orchestra.run.smithi031.stdout:3836 punch from 0x90a91dd to 0x90b6172, (0xcf95 bytes) 2024-09-16T22:17:15.576 INFO:teuthology.orchestra.run.smithi031.stdout:3838 write 0x82ae03d thru 0x82baa35 (0xc9f9 bytes) 2024-09-16T22:17:15.582 INFO:teuthology.orchestra.run.smithi031.stdout:3839 trunc from 0xde47e53 to 0xb093989 2024-09-16T22:17:15.588 INFO:teuthology.orchestra.run.smithi031.stdout:3841 read 0x96700d6 thru 0x967c213 (0xc13e bytes) 2024-09-16T22:17:15.588 INFO:teuthology.orchestra.run.smithi031.stdout:3842 punch from 0x3f5b283 to 0x3f68234, (0xcfb1 bytes) 2024-09-16T22:17:15.588 INFO:teuthology.orchestra.run.smithi031.stdout:3843 read 0x31e1ab9 thru 0x31ec7a7 (0xacef bytes) 2024-09-16T22:17:15.588 INFO:teuthology.orchestra.run.smithi031.stdout:3845 punch from 0xae291d to 0xaeb922, (0x9005 bytes) 2024-09-16T22:17:15.588 INFO:teuthology.orchestra.run.smithi031.stdout:3846 write 0xe37ff4 thru 0xe466c3 (0xe6d0 bytes) 2024-09-16T22:17:15.590 INFO:teuthology.orchestra.run.smithi031.stdout:3848 read 0x5709f82 thru 0x5710397 (0x6416 bytes) 2024-09-16T22:17:15.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.590 INFO:teuthology.orchestra.run.smithi031.stdout:3850 punch from 0x80994fb to 0x80a89a0, (0xf4a5 bytes) 2024-09-16T22:17:15.601 INFO:teuthology.orchestra.run.smithi031.stdout:3851 trunc from 0xb093989 to 0xaa58c3 2024-09-16T22:17:15.614 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.614 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.619 INFO:teuthology.orchestra.run.smithi031.stdout:3852 read 0x2ca00 thru 0x32447 (0x5a48 bytes) 2024-09-16T22:17:15.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:15.630 INFO:teuthology.orchestra.run.smithi031.stdout:3855 write 0x4033592 thru 0x4040fa5 (0xda14 bytes) 2024-09-16T22:17:15.634 INFO:teuthology.orchestra.run.smithi031.stdout:3857 punch from 0x12c9bca to 0x12cf917, (0x5d4d bytes) 2024-09-16T22:17:15.634 INFO:teuthology.orchestra.run.smithi031.stdout:3858 write 0x3b1c718 thru 0x3b1cf06 (0x7ef bytes) 2024-09-16T22:17:15.636 INFO:teuthology.orchestra.run.smithi031.stdout:3859 write 0x3bda430 thru 0x3bdaca2 (0x873 bytes) 2024-09-16T22:17:15.640 INFO:teuthology.orchestra.run.smithi031.stdout:3860 read 0xde09b5 thru 0xde52d7 (0x4923 bytes) 2024-09-16T22:17:15.648 INFO:teuthology.orchestra.run.smithi031.stdout:3861 trunc from 0x4040fa6 to 0xaad3897 2024-09-16T22:17:15.648 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:15.650+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:15.654 INFO:teuthology.orchestra.run.smithi031.stdout:3862 trunc from 0xaad3897 to 0x9e969c2 2024-09-16T22:17:15.661 INFO:teuthology.orchestra.run.smithi031.stdout:3863 trunc from 0x9e969c2 to 0x90cd9d5 2024-09-16T22:17:15.668 INFO:teuthology.orchestra.run.smithi031.stdout:3864 punch from 0x4d0ac2b to 0x4d0fd86, (0x515b bytes) 2024-09-16T22:17:15.668 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.668 INFO:teuthology.orchestra.run.smithi031.stdout:3866 read 0x1775051 thru 0x17819d3 (0xc983 bytes) 2024-09-16T22:17:15.668 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.674 INFO:teuthology.orchestra.run.smithi031.stdout:3868 write 0x9b9ee75 thru 0x9bac478 (0xd604 bytes) 2024-09-16T22:17:15.684 INFO:teuthology.orchestra.run.smithi031.stdout:3869 writesame 0x9e49297 thru 0x9e567ae data_size 0xc(0xd518 bytes) 2024-09-16T22:17:15.694 INFO:teuthology.orchestra.run.smithi031.stdout:3871 trunc from 0x9e567af to 0x20b10ba 2024-09-16T22:17:15.709 INFO:teuthology.orchestra.run.smithi031.stdout:3872 punch from 0x1c09a5e to 0x1c188d8, (0xee7a bytes) 2024-09-16T22:17:15.709 INFO:teuthology.orchestra.run.smithi031.stdout:3873 read 0x1209ea thru 0x12276a (0x1d81 bytes) 2024-09-16T22:17:15.725 INFO:teuthology.orchestra.run.smithi031.stdout:3874 writesame 0xb06d367 thru 0xb075c1b data_size 0x1(0x88b5 bytes) 2024-09-16T22:17:15.725 INFO:teuthology.orchestra.run.smithi031.stdout:3875 read 0x67f180a thru 0x67fbc7b (0xa472 bytes) 2024-09-16T22:17:15.728 INFO:teuthology.orchestra.run.smithi031.stdout:3876 trunc from 0xb075c1c to 0x839c49f 2024-09-16T22:17:15.734 INFO:teuthology.orchestra.run.smithi031.stdout:3877 write 0x17dd120 thru 0x17dd920 (0x801 bytes) 2024-09-16T22:17:15.735 INFO:teuthology.orchestra.run.smithi031.stdout:3879 read 0x299e074 thru 0x29ac85d (0xe7ea bytes) 2024-09-16T22:17:15.735 INFO:teuthology.orchestra.run.smithi031.stdout:3881 punch from 0x270afa5 to 0x270bc45, (0xca0 bytes) 2024-09-16T22:17:15.735 INFO:teuthology.orchestra.run.smithi031.stdout:3883 write 0x12db784 thru 0x12eb599 (0xfe16 bytes) 2024-09-16T22:17:15.744 INFO:teuthology.orchestra.run.smithi031.stdout:3888 trunc from 0x839c49f to 0xe90d0c3 2024-09-16T22:17:15.749 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:15.751+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:3890 punch from 0x584e0cf to 0x5856852, (0x8783 bytes) 2024-09-16T22:17:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:3893 read 0x7e2b5f6 thru 0x7e345df (0x8fea bytes) 2024-09-16T22:17:15.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:15.750 INFO:teuthology.orchestra.run.smithi031.stdout:3898 write 0x93c9ac8 thru 0x93d851d (0xea56 bytes) 2024-09-16T22:17:15.750 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.750 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.753 INFO:teuthology.orchestra.run.smithi031.stdout:3899 write 0x5fb8938 thru 0x5fc48db (0xbfa4 bytes) 2024-09-16T22:17:15.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:15.767 INFO:teuthology.orchestra.run.smithi031.stdout:3901 trunc from 0xe90d0c3 to 0x51df386 2024-09-16T22:17:15.784 INFO:teuthology.orchestra.run.smithi031.stdout:3902 trunc from 0x51df386 to 0xa016409 2024-09-16T22:17:15.790 INFO:teuthology.orchestra.run.smithi031.stdout:3903 write 0x3465bc3 thru 0x3471270 (0xb6ae bytes) 2024-09-16T22:17:15.793 INFO:teuthology.orchestra.run.smithi031.stdout:3906 read 0x44245d0 thru 0x44286ec (0x411d bytes) 2024-09-16T22:17:15.793 INFO:teuthology.orchestra.run.smithi031.stdout:3907 clone 35 order 20 su 16384 sc 3 2024-09-16T22:17:15.850 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:15.851+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:15.877 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:15.877 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:15.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:15.969 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:15.971+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:16.033 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:16.033 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:16.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:16.052 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:16.053+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:16.152 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:16.154+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:16.189 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:16.189 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:16.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:16.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:17:16.285 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:16.287+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:16.318 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:16.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:16.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:16.322 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone34 from 0xa016409 (overlap 0xaa58c3) to 0x2211c0 2024-09-16T22:17:16.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:16.363 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:16.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:16.363 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T22:17:16.364 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:16.364 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:16.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T22:17:16.370 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:16.372+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:16.454 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:16.456+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:16.489 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:16.489 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:16.496 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T22:17:16.499 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:17:16.499 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:16.514 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:16.515 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:16.524 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:16.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:16.542 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:16.543 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:16.554 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:16.556+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:16.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:16.594 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:16.639 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:16.659 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:16.657+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:16.678 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:16.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:16.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:16.738 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:16.739 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:16.739 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:16.739 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:16.744 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:16.745 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:16.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:16.756 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:16.759+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:16.858 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:16.859+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:16.908 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:16.909 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:16.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:16.944 INFO:teuthology.orchestra.run.smithi031.stdout:3908 write 0x2b6268c thru 0x2b62a18 (0x38d bytes) 2024-09-16T22:17:16.958 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:16.960+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:3909 trunc from 0xa016409 to 0x36ae99a 2024-09-16T22:17:16.974 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:16.987 INFO:teuthology.orchestra.run.smithi031.stdout:3911 write 0x987f9c0 thru 0x9885a2f (0x6070 bytes) 2024-09-16T22:17:16.992 INFO:teuthology.orchestra.run.smithi031.stdout:3912 read 0x23b501b thru 0x23bc562 (0x7548 bytes) 2024-09-16T22:17:16.992 INFO:teuthology.orchestra.run.smithi031.stdout:3913 read 0x7c585a6 thru 0x7c5db50 (0x55ab bytes) 2024-09-16T22:17:16.992 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:16.992 INFO:teuthology.orchestra.run.smithi031.stdout:3915 read 0x2671f67 thru 0x26730ee (0x1188 bytes) 2024-09-16T22:17:16.993 INFO:teuthology.orchestra.run.smithi031.stdout:3916 write 0x645bd6e thru 0x64601df (0x4472 bytes) 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.459796260000001, 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:17.036 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.459776919999999, 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.459372519999999, 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45048 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.457699075000001, 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676787000000002, 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45048 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579108+0000", 2024-09-16T22:17:17.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579110+0000", 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579116+0000", 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579120+0000", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630608+0000", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051487796000000002 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630687+0000", 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8709999999999997e-05 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630784+0000", 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7190999999999998e-05 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.45767725, 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:17.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.457646716999999, 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:17.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.453245817999999, 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:17.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:17.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.453225336999999, 2024-09-16T22:17:17.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:17.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:17.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.453204760999999, 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:17.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.453180276000001, 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:17.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.453158705, 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:17.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:17.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45147 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.447100931000001, 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051674649000000003, 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45147 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589705+0000", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589706+0000", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589708+0000", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9599999999999999e-06 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589714+0000", 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589720+0000", 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.383e-06 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641214+0000", 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051494686999999997 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641285+0000", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0795000000000002e-05 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641380+0000", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5118999999999995e-05 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:17.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.447080107000001, 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:17.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.447059775, 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:17.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.447038555999999, 2024-09-16T22:17:17.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.447017184, 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:17.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.446996345999999, 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:17.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.446975765000001, 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.446954198, 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.442307324000002, 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.127 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:17.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:17.132 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:17.130+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:17.191 INFO:teuthology.orchestra.run.smithi031.stdout:3917 punch from 0x72a5cf7 to 0x72b4775, (0xea7e bytes) 2024-09-16T22:17:17.202 INFO:teuthology.orchestra.run.smithi031.stdout:3921 writesame 0xe208ca1 thru 0xe218a35 data_size 0x3(0xfd95 bytes) 2024-09-16T22:17:17.202 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.203 INFO:teuthology.orchestra.run.smithi031.stdout:3924 read 0x9f1c286 thru 0x9f2210b (0x5e86 bytes) 2024-09-16T22:17:17.203 INFO:teuthology.orchestra.run.smithi031.stdout:3925 punch from 0x1035399 to 0x103e1e5, (0x8e4c bytes) 2024-09-16T22:17:17.203 INFO:teuthology.orchestra.run.smithi031.stdout:3927 writesame 0x58c9508 thru 0x58c9680 data_size 0x1(0x179 bytes) 2024-09-16T22:17:17.205 INFO:teuthology.orchestra.run.smithi031.stdout:3928 read 0x816c195 thru 0x817919a (0xd006 bytes) 2024-09-16T22:17:17.205 INFO:teuthology.orchestra.run.smithi031.stdout:3929 punch from 0x3425d44 to 0x342c8d2, (0x6b8e bytes) 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:3933 read 0xd447a20 thru 0xd449ba0 (0x2181 bytes) 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:3935 read 0x3c088fd thru 0x3c0e7e8 (0x5eec bytes) 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:3937 read 0x402362d thru 0x4028daf (0x5783 bytes) 2024-09-16T22:17:17.206 INFO:teuthology.orchestra.run.smithi031.stdout:3938 write 0x402f7e6 thru 0x403cef7 (0xd712 bytes) 2024-09-16T22:17:17.209 INFO:teuthology.orchestra.run.smithi031.stdout:3939 write 0xccefe10 thru 0xccf35e1 (0x37d2 bytes) 2024-09-16T22:17:17.213 INFO:teuthology.orchestra.run.smithi031.stdout:3941 punch from 0xdd17799 to 0xdd1914c, (0x19b3 bytes) 2024-09-16T22:17:17.213 INFO:teuthology.orchestra.run.smithi031.stdout:3942 writesame 0x8202bc thru 0x82319b data_size 0x32(0x2ee0 bytes) 2024-09-16T22:17:17.224 INFO:teuthology.orchestra.run.smithi031.stdout:3943 punch from 0xabfd39b to 0xac03f9b, (0x6c00 bytes) 2024-09-16T22:17:17.224 INFO:teuthology.orchestra.run.smithi031.stdout:3944 write 0xce7c4c3 thru 0xce85078 (0x8bb6 bytes) 2024-09-16T22:17:17.229 INFO:teuthology.orchestra.run.smithi031.stdout:3945 write 0x6e1105b thru 0x6e1bf3f (0xaee5 bytes) 2024-09-16T22:17:17.229 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:17.231+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:17:17.239 INFO:teuthology.orchestra.run.smithi031.stdout:3951 trunc from 0xe218a36 to 0xa6a8eb1 2024-09-16T22:17:17.257 INFO:teuthology.orchestra.run.smithi031.stdout:3952 write 0x4dd301b thru 0x4dd5c72 (0x2c58 bytes) 2024-09-16T22:17:17.262 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.262 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:17.262 INFO:teuthology.orchestra.run.smithi031.stdout:3956 writesame 0x1d61cbf thru 0x1d69b7d data_size 0x1(0x7ebf bytes) 2024-09-16T22:17:17.284 INFO:teuthology.orchestra.run.smithi031.stdout:3957 punch from 0x6a7e7d8 to 0x6a86a8b, (0x82b3 bytes) 2024-09-16T22:17:17.284 INFO:teuthology.orchestra.run.smithi031.stdout:3959 clone 36 order 18 su 2048 sc 9 2024-09-16T22:17:17.301 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:17.301 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:17.302 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:17.302 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:17.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:17.330 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:17.331+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:17.431 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:17.433+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:17.460 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:17.460 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:17.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:17.553 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:17.555+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:17.617 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.191342625000004, 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.356537573000001, 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.356516734, 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.356493493999999, 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.355791609000001, 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.352659161999998, 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.352249552, 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.351963375, 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.351666151, 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.350877908000001, 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.350763139000001, 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.350511921999999, 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.350203564000001, 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.350164627000002, 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.349879207000001, 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.344027703999998, 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.344003306000001, 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.343952655999999, 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.343918959, 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.343807988999998, 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:17.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:17.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:17.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:17.708 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:17.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:17.710 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:17.711+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:17.810 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:17.812+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:17.816 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:17.816 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:17.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:17.911 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:17.913+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:17.967 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:17.967 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:17.967 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:17.967 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:17.967 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:17.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:18.018 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:18.020+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42585 3.7 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.925008270999999, 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052149483000000003, 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42585 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155589+0000", 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.155591+0000", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.573e-06 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170506+0000", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01491556 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170523+0000", 2024-09-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6545000000000002e-05 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170530+0000", 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170624+0000", 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3875999999999994e-05 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170759+0000", 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013552799999999999 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 6", 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171081+0000", 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032176299999999998 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207149+0000", 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036067812999999997 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207618+0000", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046936500000000002 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207689+0000", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0827999999999993e-05 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207739+0000", 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9601999999999997e-05 2024-09-16T22:17:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:42587 3.8 3:1432c3a7:::rbd_data.11cd49ca855d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 5767168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.909948894999999, 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037194235999999999, 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 42587 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170648+0000", 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.170649+0000", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.096e-06 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181027+0000", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010377347 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181047+0000", 2024-09-16T22:17:18.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0217999999999999e-05 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181057+0000", 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0142e-05 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181119+0000", 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2163999999999996e-05 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181293+0000", 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017324699999999999 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.181934+0000", 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064154700000000002 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207675+0000", 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025741327000000001 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207750+0000", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4301000000000006e-05 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207804+0000", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4564000000000001e-05 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207843+0000", 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8282999999999999e-05 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59223 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6328320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.225198283000001, 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034753091999999999, 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59223 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855399+0000", 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855484+0000", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5171000000000002e-05 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855487+0000", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855494+0000", 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5059999999999998e-06 2024-09-16T22:17:18.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855499+0000", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9250000000000003e-06 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890044+0000", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034544856999999998 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890094+0000", 2024-09-16T22:17:18.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9632999999999997e-05 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890152+0000", 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8329999999999999e-05 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59224 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6373376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.225088702000001, 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034745103999999999, 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59224 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855509+0000", 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855695+0000", 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000186205 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855697+0000", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.481e-06 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855702+0000", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.855706+0000", 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8510000000000001e-06 2024-09-16T22:17:18.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890164+0000", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034458216 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890204+0000", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9756999999999999e-05 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890254+0000", 2024-09-16T22:17:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9916000000000001e-05 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59229 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6377472~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.224425992, 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034688831000000003, 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59229 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856171+0000", 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856260+0000", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8486000000000006e-05 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856266+0000", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6000000000000003e-06 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856273+0000", 2024-09-16T22:17:18.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5629999999999997e-06 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.856280+0000", 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8090000000000004e-06 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890761+0000", 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034480982 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890802+0000", 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1625000000000002e-05 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.890860+0000", 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7766000000000002e-05 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59262 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6684672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:17:18.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.220061576999999, 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034901661, 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59262 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:17:18.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860536+0000", 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860744+0000", 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208519 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860747+0000", 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5210000000000001e-06 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860752+0000", 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860758+0000", 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8479999999999997e-06 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895226+0000", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034467822000000002 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895320+0000", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4729999999999996e-05 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895437+0000", 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011711599999999999 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59263 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6656000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.219815265000001, 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034894085999999998, 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59263 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860782+0000", 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860783+0000", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2200000000000004e-07 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860785+0000", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0839999999999999e-06 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860791+0000", 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8390000000000003e-06 2024-09-16T22:17:18.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860796+0000", 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8139999999999997e-06 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895487+0000", 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034690152000000002 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895579+0000", 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2514000000000002e-05 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895676+0000", 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7060999999999998e-05 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59264 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6701056~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:17:18.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.219792333000001, 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034991059999999997, 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59264 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:17:18.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.860805+0000", 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862891+0000", 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002086409 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862895+0000", 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0110000000000002e-06 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862904+0000", 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0920000000000004e-06 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.862911+0000", 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4049999999999997e-06 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895691+0000", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032779642999999997 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895744+0000", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3767000000000002e-05 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.895796+0000", 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1733000000000001e-05 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59275 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6754304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217449883, 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034872215999999998, 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59275 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863147+0000", 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863148+0000", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863149+0000", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1000000000000001e-06 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863152+0000", 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0369999999999999e-06 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863154+0000", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.897823+0000", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034668869999999997 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.897915+0000", 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1735000000000002e-05 2024-09-16T22:17:18.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898020+0000", 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104777 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59276 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6799360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217440056000001, 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034967300999999999, 2024-09-16T22:17:18.123 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59276 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863157+0000", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863159+0000", 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1629999999999999e-06 2024-09-16T22:17:18.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863162+0000", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249999999999999e-06 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863164+0000", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5849999999999998e-06 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898033+0000", 2024-09-16T22:17:18.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034868561999999999 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898076+0000", 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3163e-05 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898125+0000", 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8683000000000001e-05 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59277 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6770688~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217429533000001, 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035191287000000002, 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59277 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863168+0000", 2024-09-16T22:17:18.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863175+0000", 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0360000000000001e-06 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863178+0000", 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4340000000000001e-06 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863181+0000", 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.695e-06 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898161+0000", 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034980306000000003 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898256+0000", 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4986000000000002e-05 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898359+0000", 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102606 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59278 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6815744~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217412770999999, 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035275754999999999, 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59278 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863185+0000", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2100000000000001e-07 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863186+0000", 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.203e-06 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863189+0000", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.948e-06 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863191+0000", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5450000000000002e-06 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898372+0000", 2024-09-16T22:17:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035180077999999997 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898411+0000", 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9067999999999999e-05 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898460+0000", 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9691999999999998e-05 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59279 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6787072~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217402961000001, 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03549563, 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59279 2024-09-16T22:17:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863194+0000", 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863195+0000", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.48e-07 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863196+0000", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249999999999999e-06 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863199+0000", 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2059999999999999e-06 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863204+0000", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7779999999999998e-06 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898497+0000", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035293195999999999 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898592+0000", 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5228000000000004e-05 2024-09-16T22:17:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898690+0000", 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7948999999999993e-05 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59280 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6832128~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217377583000001, 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035629223000000002, 2024-09-16T22:17:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59280 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863220+0000", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863222+0000", 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0899999999999999e-06 2024-09-16T22:17:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863228+0000", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3630000000000002e-06 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863235+0000", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7959999999999998e-06 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898723+0000", 2024-09-16T22:17:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035488820999999997 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898775+0000", 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1425999999999999e-05 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898849+0000", 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4175e-05 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59281 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6803456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217355155, 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035848440000000002, 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59281 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863242+0000", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863243+0000", 2024-09-16T22:17:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863245+0000", 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.074e-06 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863251+0000", 2024-09-16T22:17:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8539999999999999e-06 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863258+0000", 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1299999999999998e-06 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898889+0000", 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035631384000000002 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.898987+0000", 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8062999999999999e-05 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899091+0000", 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103498 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59282 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6848512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217333132, 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035937208999999998, 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59282 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863264+0000", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863265+0000", 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-16T22:17:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863266+0000", 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863275+0000", 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2560000000000002e-06 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863281+0000", 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0100000000000001e-06 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899106+0000", 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035825203999999999 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899148+0000", 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2233e-05 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899201+0000", 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3340000000000001e-05 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59283 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6819840~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217303451999999, 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036143469999999997, 2024-09-16T22:17:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59283 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863294+0000", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863296+0000", 2024-09-16T22:17:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1469999999999999e-06 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863304+0000", 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7049999999999999e-06 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863310+0000", 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6949999999999997e-06 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899239+0000", 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035928845000000001 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899335+0000", 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5872000000000006e-05 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899437+0000", 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102746 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59284 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6864896~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217279185999999, 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036222813999999999, 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59284 2024-09-16T22:17:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863318+0000", 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863519+0000", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020047899999999999 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863522+0000", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863529+0000", 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9090000000000003e-06 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863535+0000", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6080000000000001e-06 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899451+0000", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035915646000000002 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899491+0000", 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0188000000000002e-05 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899541+0000", 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9824000000000003e-05 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59285 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6836224~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217052199000001, 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036230115, 2024-09-16T22:17:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59285 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863545+0000", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863546+0000", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863547+0000", 2024-09-16T22:17:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863556+0000", 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6819999999999992e-06 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863561+0000", 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899579+0000", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036018224000000001 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899672+0000", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2588000000000002e-05 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899775+0000", 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103447 2024-09-16T22:17:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:59286 3.1b 3:dba7b16d:::rbd_data.11cd13540b0c.0000000000000004:1a [read 6881280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.217027430000002, 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036317443999999997, 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59286 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.863570+0000", 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868940+0000", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053705530000000001 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868947+0000", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3589999999999996e-06 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868956+0000", 2024-09-16T22:17:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1700000000000003e-06 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.868964+0000", 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1489999999999994e-06 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899789+0000", 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030824579000000001 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899834+0000", 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5478000000000001e-05 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:58.899887+0000", 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3155999999999997e-05 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.173 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:18.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:18.175 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:18.177+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:18.275 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:18.277+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:18.322 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:18.322 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:18.322 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:18.322 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:18.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:18.339 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone35 from 0xa6a8eb1 (overlap 0x36ae99a) to 0x2bbee14 2024-09-16T22:17:18.376 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:18.378+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:18.440 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:18.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:18.440 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:18.440 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:18.440 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:18.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:18.492 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:18.494+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:18.518 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T22:17:18.539 INFO:teuthology.orchestra.run.smithi031.stdout:3960 write 0xeca1b70 thru 0xecb197f (0xfe10 bytes) 2024-09-16T22:17:18.551 INFO:teuthology.orchestra.run.smithi031.stdout:3961 write 0x7ae85ff thru 0x7aee7e8 (0x61ea bytes) 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.052666379, 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.051120601999999, 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.051102522000001, 2024-09-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.05108596, 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.051069472, 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.0510524, 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.051035894, 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.051006102000001, 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046673845000001, 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046651583999999, 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046630589999999, 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046609842000001, 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046589539999999, 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046569537, 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046549411999999, 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046529817, 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046508895000001, 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046488107, 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046466446, 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.046437356, 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:18.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.648 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:18.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:18.651 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:18.651+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:3962 read 0x2de76c9 thru 0x2df37b7 (0xc0ef bytes) 2024-09-16T22:17:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:3963 read 0x7db27d9 thru 0x7dbaabc (0x82e4 bytes) 2024-09-16T22:17:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:3964 punch from 0x857c5c7 to 0x857f939, (0x3372 bytes) 2024-09-16T22:17:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:3965 read 0x9d17622 thru 0x9d2212c (0xab0b bytes) 2024-09-16T22:17:18.652 INFO:teuthology.orchestra.run.smithi031.stdout:3966 read 0xa751c47 thru 0xa754a7a (0x2e34 bytes) 2024-09-16T22:17:18.652 INFO:teuthology.orchestra.run.smithi031.stdout:3967 read 0x173f547 thru 0x1742cfe (0x37b8 bytes) 2024-09-16T22:17:18.652 INFO:teuthology.orchestra.run.smithi031.stdout:3968 write 0x8fd5758 thru 0x8fddb83 (0x842c bytes) 2024-09-16T22:17:18.652 INFO:teuthology.orchestra.run.smithi031.stdout:3969 write 0x877843 thru 0x87d64f (0x5e0d bytes) 2024-09-16T22:17:18.718 INFO:teuthology.orchestra.run.smithi031.stdout:3970 write 0x7d416d4 thru 0x7d43676 (0x1fa3 bytes) 2024-09-16T22:17:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:3971 punch from 0x65eea07 to 0x65f13d2, (0x29cb bytes) 2024-09-16T22:17:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:3972 read 0x8f3ab8d thru 0x8f41878 (0x6cec bytes) 2024-09-16T22:17:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:3973 punch from 0x6c42950 to 0x6c44715, (0x1dc5 bytes) 2024-09-16T22:17:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:3974 trunc from 0xecb1980 to 0xec4248b 2024-09-16T22:17:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:3976 read 0x43f1e8b thru 0x43f9ac1 (0x7c37 bytes) 2024-09-16T22:17:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:3978 writesame 0xa26749a thru 0xa26d458 data_size 0x1(0x5fbf bytes) 2024-09-16T22:17:18.750 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:18.752+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:3979 punch from 0x1400971 to 0x140f6bd, (0xed4c bytes) 2024-09-16T22:17:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:3980 write 0x1e51be3 thru 0x1e5cd1b (0xb139 bytes) 2024-09-16T22:17:18.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:18.807 INFO:teuthology.orchestra.run.smithi031.stdout:3981 write 0x221575 thru 0x224648 (0x30d4 bytes) 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135146 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818051+0000", 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012659867999999999, 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012681026, 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135146 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818051+0000", 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818051+0000", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818052+0000", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818055+0000", 2024-09-16T22:17:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.41e-06 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818064+0000", 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0939999999999997e-06 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818071+0000", 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8650000000000003e-06 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135451 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820310+0000", 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010400708999999999, 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010479821, 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135451 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820310+0000", 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820310+0000", 2024-09-16T22:17:18.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820311+0000", 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820312+0000", 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820318+0000", 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8209999999999998e-06 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820323+0000", 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135170 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818080+0000", 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012630943, 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01276415, 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135170 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818080+0000", 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818080+0000", 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818080+0000", 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.63e-07 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818082+0000", 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818091+0000", 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3280000000000006e-06 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818097+0000", 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8340000000000004e-06 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135476 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820329+0000", 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010382251, 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010564667, 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135476 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820329+0000", 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820329+0000", 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820329+0000", 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4799999999999999e-07 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820336+0000", 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6649999999999997e-06 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820341+0000", 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6679999999999998e-06 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820346+0000", 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6070000000000001e-06 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135172 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818104+0000", 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012607274, 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012835713, 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135172 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818104+0000", 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818104+0000", 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818104+0000", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818106+0000", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818113+0000", 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2389999999999996e-06 2024-09-16T22:17:18.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818119+0000", 2024-09-16T22:17:18.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135477 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820353+0000", 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010358174, 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010634678, 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135477 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820353+0000", 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820353+0000", 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820353+0000", 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6899999999999998e-07 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820355+0000", 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2340000000000001e-06 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820360+0000", 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.831e-06 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820364+0000", 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1640000000000003e-06 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135192 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818445+0000", 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012265999, 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012589359, 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135192 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818445+0000", 2024-09-16T22:17:18.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818445+0000", 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818446+0000", 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1099999999999995e-07 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818450+0000", 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9890000000000003e-06 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818461+0000", 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1884e-05 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818469+0000", 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135486 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820371+0000", 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010340222, 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010710572999999999, 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135486 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820371+0000", 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820371+0000", 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820371+0000", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820373+0000", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1090000000000001e-06 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820378+0000", 2024-09-16T22:17:18.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2909999999999998e-06 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820383+0000", 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6419999999999998e-06 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135208 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818481+0000", 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012230293999999999, 2024-09-16T22:17:18.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012646564000000001, 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135208 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818481+0000", 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818481+0000", 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818481+0000", 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818484+0000", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6929999999999999e-06 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818491+0000", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1199999999999996e-06 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818497+0000", 2024-09-16T22:17:18.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999999e-06 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135495 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820548+0000", 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010162503999999999, 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010626605000000001, 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.867 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135495 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820548+0000", 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820548+0000", 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820549+0000", 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820552+0000", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.354e-06 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820560+0000", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6469999999999998e-06 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820566+0000", 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.229e-06 2024-09-16T22:17:18.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135209 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818504+0000", 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.012206722999999999, 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012717427999999999, 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135209 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818504+0000", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818504+0000", 2024-09-16T22:17:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818505+0000", 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818507+0000", 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0760000000000001e-06 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818514+0000", 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8820000000000003e-06 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818519+0000", 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2209999999999996e-06 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135496 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.874 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820575+0000", 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010135508999999999, 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010692223000000001, 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135496 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820575+0000", 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820575+0000", 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820576+0000", 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820578+0000", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820583+0000", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.614e-06 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820588+0000", 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6099999999999999e-06 2024-09-16T22:17:18.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135225 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818796+0000", 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011915289000000001, 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012517781, 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135225 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818796+0000", 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818796+0000", 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818796+0000", 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.99e-07 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818800+0000", 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.411e-06 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818808+0000", 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0949999999999996e-06 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818814+0000", 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.285e-06 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135512 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820600+0000", 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010110965, 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01075943, 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135512 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820600+0000", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820600+0000", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820600+0000", 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:18.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820603+0000", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5059999999999998e-06 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820609+0000", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4949999999999996e-06 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820615+0000", 2024-09-16T22:17:18.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4179999999999996e-06 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135226 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818823+0000", 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011888015, 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012584579, 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.885 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135226 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818823+0000", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818823+0000", 2024-09-16T22:17:18.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818823+0000", 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818826+0000", 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.712e-06 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818842+0000", 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6181e-05 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818848+0000", 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6949999999999997e-06 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135513 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.888 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820622+0000", 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010089177, 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010832921000000001, 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135513 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820622+0000", 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820622+0000", 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820622+0000", 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820825+0000", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202747 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820839+0000", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3869999999999999e-05 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820845+0000", 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3380000000000001e-06 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135227 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.818856+0000", 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.01185462, 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012644173999999999, 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135227 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818856+0000", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818856+0000", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818857+0000", 2024-09-16T22:17:18.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818859+0000", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3029999999999998e-06 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.818866+0000", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.545e-06 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819161+0000", 2024-09-16T22:17:18.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029513899999999998 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135528 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820856+0000", 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0098549229999999998, 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010694409, 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.895 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135528 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820856+0000", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820856+0000", 2024-09-16T22:17:18.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820857+0000", 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6400000000000002e-07 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820859+0000", 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820865+0000", 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8429999999999998e-06 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820870+0000", 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7929999999999997e-06 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135251 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819173+0000", 2024-09-16T22:17:18.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011538217, 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012425422, 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135251 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819173+0000", 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819173+0000", 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819173+0000", 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5499999999999998e-07 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819181+0000", 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7849999999999992e-06 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819189+0000", 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1159999999999999e-06 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819195+0000", 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4330000000000003e-06 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135529 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820877+0000", 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0098339459999999997, 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010767654999999999, 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.902 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135529 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820877+0000", 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820877+0000", 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820877+0000", 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2e-07 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820879+0000", 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0810000000000001e-06 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820885+0000", 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9689999999999999e-06 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820890+0000", 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6589999999999998e-06 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135253 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819203+0000", 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011507719, 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012487882, 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135253 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819203+0000", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819203+0000", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819204+0000", 2024-09-16T22:17:18.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819206+0000", 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2819999999999999e-06 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819217+0000", 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1331e-05 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819223+0000", 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.468e-06 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135530 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820897+0000", 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0098137859999999997, 2024-09-16T22:17:18.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010839591000000001, 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135530 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820897+0000", 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820897+0000", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820898+0000", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820900+0000", 2024-09-16T22:17:18.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820905+0000", 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3129999999999998e-06 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820909+0000", 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4649999999999996e-06 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135254 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819230+0000", 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011480575, 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012553185, 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135254 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819230+0000", 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819230+0000", 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819231+0000", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819233+0000", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1730000000000002e-06 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819505+0000", 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027209000000000001 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819512+0000", 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7619999999999998e-06 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135556 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.822434+0000", 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0082769750000000007, 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094001759999999997, 2024-09-16T22:17:18.916 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135556 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822434+0000", 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822434+0000", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822435+0000", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5799999999999998e-07 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822439+0000", 2024-09-16T22:17:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2629999999999997e-06 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822449+0000", 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6949999999999995e-06 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822461+0000", 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2507000000000001e-05 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135273 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819523+0000", 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011187842999999999, 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012358644, 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135273 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.920 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819523+0000", 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819523+0000", 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819524+0000", 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.44e-07 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819527+0000", 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.41e-06 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819534+0000", 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4189999999999996e-06 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819547+0000", 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2309e-05 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135566 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.822473+0000", 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0082373979999999999, 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094550450000000005, 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135566 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822473+0000", 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822473+0000", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822474+0000", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-07 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822476+0000", 2024-09-16T22:17:18.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779999999999999e-06 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822483+0000", 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8530000000000003e-06 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.822488+0000", 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1989999999999997e-06 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135274 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819555+0000", 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011155750000000001, 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012420186, 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135274 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819555+0000", 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819555+0000", 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819556+0000", 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819559+0000", 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9809999999999999e-06 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819565+0000", 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.81e-06 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819571+0000", 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5289999999999999e-06 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135275 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819578+0000", 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011132342999999999, 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012442533, 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135275 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819578+0000", 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819578+0000", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819579+0000", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819859+0000", 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028036100000000002 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819868+0000", 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8410000000000006e-06 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819874+0000", 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2770000000000002e-06 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135300 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819886+0000", 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010825248000000001, 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01218144, 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135300 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819886+0000", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819886+0000", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819886+0000", 2024-09-16T22:17:18.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819889+0000", 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8119999999999999e-06 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819896+0000", 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5180000000000001e-06 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819901+0000", 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1150000000000002e-06 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135301 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819908+0000", 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010803244, 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012205690999999999, 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135301 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819908+0000", 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819908+0000", 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819908+0000", 2024-09-16T22:17:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819910+0000", 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819916+0000", 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3400000000000003e-06 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819921+0000", 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8339999999999998e-06 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135321 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.819928+0000", 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010782879, 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012232252000000001, 2024-09-16T22:17:18.941 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135321 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819928+0000", 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819928+0000", 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819928+0000", 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819930+0000", 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.928e-06 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.819936+0000", 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820001+0000", 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5204999999999997e-05 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135322 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820014+0000", 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010696423, 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012192026, 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135322 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820014+0000", 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820014+0000", 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820015+0000", 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0699999999999997e-07 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820018+0000", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-06 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820026+0000", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0530000000000001e-06 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820031+0000", 2024-09-16T22:17:18.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000003e-06 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135347 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820057+0000", 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010654195999999999, 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012196218, 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135347 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820057+0000", 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820057+0000", 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820057+0000", 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3200000000000005e-07 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820060+0000", 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5220000000000002e-06 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820066+0000", 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0000000000000002e-06 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820071+0000", 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135348 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820078+0000", 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010632744, 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012221825, 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135348 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820078+0000", 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820078+0000", 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820079+0000", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820081+0000", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129999999999999e-06 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820087+0000", 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:17:18.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820091+0000", 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4510000000000002e-06 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135349 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820097+0000", 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010613516999999999, 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012248624, 2024-09-16T22:17:18.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135349 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820097+0000", 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820097+0000", 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820098+0000", 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820104+0000", 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000002e-06 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820111+0000", 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3350000000000003e-06 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820116+0000", 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4970000000000001e-06 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135369 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820123+0000", 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010587618, 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012309911, 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135369 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820123+0000", 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820123+0000", 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820124+0000", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820125+0000", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820131+0000", 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8000000000000004e-06 2024-09-16T22:17:18.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820136+0000", 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8040000000000001e-06 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135370 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820142+0000", 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010568935999999999, 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012338549000000001, 2024-09-16T22:17:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135370 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820142+0000", 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820142+0000", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820142+0000", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820144+0000", 2024-09-16T22:17:18.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7039999999999999e-06 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820150+0000", 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6219999999999997e-06 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820155+0000", 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9359999999999998e-06 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135372 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820160+0000", 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010550469, 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012370582999999999, 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135372 2024-09-16T22:17:18.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820160+0000", 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820160+0000", 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820161+0000", 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820162+0000", 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.699e-06 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820169+0000", 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9439999999999999e-06 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820174+0000", 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000001e-06 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135396 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820180+0000", 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010530578000000001, 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012397778999999999, 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135396 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.971 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820180+0000", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820180+0000", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820181+0000", 2024-09-16T22:17:18.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2300000000000002e-07 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820183+0000", 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0059999999999999e-06 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820189+0000", 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1009999999999997e-06 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820193+0000", 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4719999999999997e-06 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135397 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820200+0000", 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010510587, 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01242831, 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135397 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820200+0000", 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820200+0000", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820201+0000", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820203+0000", 2024-09-16T22:17:18.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820208+0000", 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8880000000000002e-06 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820214+0000", 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5130000000000004e-06 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135398 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820221+0000", 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010490107, 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012454377000000001, 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135398 2024-09-16T22:17:18.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820221+0000", 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820221+0000", 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820221+0000", 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.51e-07 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820223+0000", 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0470000000000001e-06 2024-09-16T22:17:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820229+0000", 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7939999999999999e-06 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820234+0000", 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135414 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820239+0000", 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0104717, 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012485229, 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135414 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820239+0000", 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820239+0000", 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820239+0000", 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4200000000000002e-07 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820241+0000", 2024-09-16T22:17:18.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7069999999999999e-06 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820247+0000", 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4550000000000003e-06 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820251+0000", 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3699999999999997e-06 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135415 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820256+0000", 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010454509000000001, 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012515333999999999, 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135415 2024-09-16T22:17:18.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820256+0000", 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820256+0000", 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820257+0000", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6100000000000002e-07 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820258+0000", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6819999999999999e-06 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820264+0000", 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3929999999999999e-06 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820268+0000", 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3019999999999996e-06 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135432 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820273+0000", 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010437558, 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012548314, 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135432 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820273+0000", 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820273+0000", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820274+0000", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820275+0000", 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7120000000000001e-06 2024-09-16T22:17:18.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820281+0000", 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.31e-06 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820285+0000", 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9559999999999999e-06 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:135450 3.1 3.e60d8301 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.820290+0000", 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.010420532, 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012583067, 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135450 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:18.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820290+0000", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820290+0000", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820291+0000", 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:18.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820293+0000", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0049999999999999e-06 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820298+0000", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5520000000000004e-06 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.820303+0000", 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8210000000000001e-06 2024-09-16T22:17:18.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 45 2024-09-16T22:17:18.996 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:18.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:18.999 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:18.999+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:18.999 INFO:teuthology.orchestra.run.smithi031.stdout:3983 read 0x3d0b9a3 thru 0x3d14f36 (0x9594 bytes) 2024-09-16T22:17:18.999 INFO:teuthology.orchestra.run.smithi031.stdout:3985 writesame 0x80ab1f3 thru 0x80b8cb8 data_size 0x2(0xdac6 bytes) 2024-09-16T22:17:18.999 INFO:teuthology.orchestra.run.smithi031.stdout:3986 punch from 0x8a55e6 to 0x8b3952, (0xe36c bytes) 2024-09-16T22:17:18.999 INFO:teuthology.orchestra.run.smithi031.stdout:3988 write 0xd7fa8c2 thru 0xd80982f (0xef6e bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3990 read 0xd769d56 thru 0xd7768a4 (0xcb4f bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3992 read 0x14e1947 thru 0x14eeea7 (0xd561 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3994 read 0x4f51a4b thru 0x4f60dc3 (0xf379 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3995 read 0x5f14673 thru 0x5f22971 (0xe2ff bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3996 trunc from 0xec4248b to 0xd80d576 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:3998 punch from 0x7d6bad1 to 0x7d6db44, (0x2073 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:4000 write 0x2d5057d thru 0x2d56c63 (0x66e7 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:4002 read 0x18af77 thru 0x18c7d7 (0x1861 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:4004 write 0x258fc70 thru 0x2590092 (0x423 bytes) 2024-09-16T22:17:19.000 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:19.001 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:19.001 INFO:teuthology.orchestra.run.smithi031.stdout:4005 write 0x46296ab thru 0x4634e4a (0xb7a0 bytes) 2024-09-16T22:17:19.001 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:19.002 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:19.002 INFO:teuthology.orchestra.run.smithi031.stdout:4006 trunc from 0xd80d576 to 0x53dbb08 2024-09-16T22:17:19.002 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:19.002 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:19.011 INFO:teuthology.orchestra.run.smithi031.stdout:4007 writesame 0x2d9fc27 thru 0x2da7cb7 data_size 0x1b(0x8091 bytes) 2024-09-16T22:17:19.012 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:19.012 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi031.stdout:4009 punch from 0x8a57dc to 0x8a989f, (0x40c3 bytes) 2024-09-16T22:17:19.028 INFO:teuthology.orchestra.run.smithi031.stdout:4010 write 0xa281761 thru 0xa28bf01 (0xa7a1 bytes) 2024-09-16T22:17:19.031 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:19.032 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:19.041 INFO:teuthology.orchestra.run.smithi031.stdout:4011 trunc from 0xa28bf02 to 0xa087b6a 2024-09-16T22:17:19.052 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:19.052 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:19.053 INFO:teuthology.orchestra.run.smithi031.stdout:4012 punch from 0x7fb8f0e to 0x7fbdf44, (0x5036 bytes) 2024-09-16T22:17:19.053 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.054 INFO:teuthology.orchestra.run.smithi031.stdout:4014 read 0x2cb1e19 thru 0x2cc0f6f (0xf157 bytes) 2024-09-16T22:17:19.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.056 INFO:teuthology.orchestra.run.smithi031.stdout:4017 punch from 0x4ff45b8 to 0x4ffe427, (0x9e6f bytes) 2024-09-16T22:17:19.062 INFO:teuthology.orchestra.run.smithi031.stdout:4018 write 0xa8a65a7 thru 0xa8a65eb (0x45 bytes) 2024-09-16T22:17:19.064 INFO:teuthology.orchestra.run.smithi031.stdout:4019 read 0x4944880 thru 0x49497b9 (0x4f3a bytes) 2024-09-16T22:17:19.065 INFO:teuthology.orchestra.run.smithi031.stdout:4020 write 0x3d883fb thru 0x3d903f7 (0x7ffd bytes) 2024-09-16T22:17:19.073 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:19.073 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.076 INFO:teuthology.orchestra.run.smithi031.stdout:4022 writesame 0x1fe5047 thru 0x1fecf9d data_size 0x1(0x7f57 bytes) 2024-09-16T22:17:19.098 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:19.099+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:19.107 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:19.128 INFO:teuthology.orchestra.run.smithi031.stdout:4023 read 0x42f14d0 thru 0x42f9bac (0x86dd bytes) 2024-09-16T22:17:19.128 INFO:teuthology.orchestra.run.smithi031.stdout:4024 read 0x27d7be8 thru 0x27e3a11 (0xbe2a bytes) 2024-09-16T22:17:19.137 INFO:teuthology.orchestra.run.smithi031.stdout:4027 trunc from 0xa8a65ec to 0x4ff7074 2024-09-16T22:17:19.157 INFO:teuthology.orchestra.run.smithi031.stdout:4028 read 0x85b036 thru 0x86ad06 (0xfcd1 bytes) 2024-09-16T22:17:19.165 INFO:teuthology.orchestra.run.smithi031.stdout:4031 write 0x6b2a39e thru 0x6b38b10 (0xe773 bytes) 2024-09-16T22:17:19.173 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.173 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:19.183 INFO:teuthology.orchestra.run.smithi031.stdout:4032 trunc from 0x6b38b11 to 0x575e790 2024-09-16T22:17:19.198 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:19.200+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:19.199 INFO:teuthology.orchestra.run.smithi031.stdout:4036 write 0x5afe067 thru 0x5b096c8 (0xb662 bytes) 2024-09-16T22:17:19.211 INFO:teuthology.orchestra.run.smithi031.stdout:4037 read 0x59d738b thru 0x59e724c (0xfec2 bytes) 2024-09-16T22:17:19.211 INFO:teuthology.orchestra.run.smithi031.stdout:4038 read 0x32dd67d thru 0x32eb382 (0xdd06 bytes) 2024-09-16T22:17:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:4040 write 0xaf1af11 thru 0xaf2653c (0xb62c bytes) 2024-09-16T22:17:19.237 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.237 INFO:teuthology.orchestra.run.smithi031.stdout:4042 read 0x16909fa thru 0x169a4f7 (0x9afe bytes) 2024-09-16T22:17:19.243 INFO:teuthology.orchestra.run.smithi031.stdout:4043 trunc from 0xaf2653d to 0xd6e0f23 2024-09-16T22:17:19.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:17:19.262 INFO:teuthology.orchestra.run.smithi031.stdout:4044 trunc from 0xd6e0f23 to 0x2d887bc 2024-09-16T22:17:19.284 INFO:teuthology.orchestra.run.smithi031.stdout:4045 punch from 0x1982a16 to 0x1991eb9, (0xf4a3 bytes) 2024-09-16T22:17:19.297 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.297 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.299 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:19.301+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:19.303 INFO:teuthology.orchestra.run.smithi031.stdout:4046 write 0xeb18771 thru 0xeb28766 (0xfff6 bytes) 2024-09-16T22:17:19.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:19.314 INFO:teuthology.orchestra.run.smithi031.stdout:4047 punch from 0x7c457fb to 0x7c4fa2e, (0xa233 bytes) 2024-09-16T22:17:19.314 INFO:teuthology.orchestra.run.smithi031.stdout:4052 punch from 0x8b41307 to 0x8b46a6a, (0x5763 bytes) 2024-09-16T22:17:19.314 INFO:teuthology.orchestra.run.smithi031.stdout:4053 writesame 0xd1cc20f thru 0xd1db7ff data_size 0x1f(0xf5f1 bytes) 2024-09-16T22:17:19.323 INFO:teuthology.orchestra.run.smithi031.stdout:4054 write 0x8206fc5 thru 0x820a86f (0x38ab bytes) 2024-09-16T22:17:19.333 INFO:teuthology.orchestra.run.smithi031.stdout:4055 write 0x9da67f5 thru 0x9daa0f5 (0x3901 bytes) 2024-09-16T22:17:19.348 INFO:teuthology.orchestra.run.smithi031.stdout:4056 read 0x735abbf thru 0x735c60e (0x1a50 bytes) 2024-09-16T22:17:19.349 INFO:teuthology.orchestra.run.smithi031.stdout:4057 trunc from 0xeb28767 to 0xdf554cd 2024-09-16T22:17:19.361 INFO:teuthology.orchestra.run.smithi031.stdout:4058 write 0xa6d1a15 thru 0xa6d6b1a (0x5106 bytes) 2024-09-16T22:17:19.373 INFO:teuthology.orchestra.run.smithi031.stdout:4060 punch from 0x13ca5f7 to 0x13cb114, (0xb1d bytes) 2024-09-16T22:17:19.373 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:19.373 INFO:teuthology.orchestra.run.smithi031.stdout:4062 write 0x17aac33 thru 0x17b03ee (0x57bc bytes) 2024-09-16T22:17:19.401 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:19.401+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi031.stdout:4063 writesame 0x5995fe2 thru 0x5999358 data_size 0x1f(0x3377 bytes) 2024-09-16T22:17:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:4064 write 0x7d8237f thru 0x7d866cd (0x434f bytes) 2024-09-16T22:17:19.442 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.442 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:19.454 INFO:teuthology.orchestra.run.smithi031.stdout:4065 punch from 0x85997b2 to 0x859e513, (0x4d61 bytes) 2024-09-16T22:17:19.455 INFO:teuthology.orchestra.run.smithi031.stdout:4066 read 0x5733e40 thru 0x5742c9b (0xee5c bytes) 2024-09-16T22:17:19.455 INFO:teuthology.orchestra.run.smithi031.stdout:4067 writesame 0x17a2155 thru 0x17af5ae data_size 0x16(0xd45a bytes) 2024-09-16T22:17:19.460 INFO:teuthology.orchestra.run.smithi031.stdout:4068 read 0xbb09b thru 0xbfe5e (0x4dc4 bytes) 2024-09-16T22:17:19.466 INFO:teuthology.orchestra.run.smithi031.stdout:4072 trunc from 0xdf554cd to 0xaeac13e 2024-09-16T22:17:19.478 INFO:teuthology.orchestra.run.smithi031.stdout:4073 clone 37 order 19 su 65536 sc 12 2024-09-16T22:17:19.501 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:19.502+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:19.597 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.597 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.601 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:19.603+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:19.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:19.703 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:19.703+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:19.709 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.709 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:19.803 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:19.805+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:19.827 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.827 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:19.919 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:19.921+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:19.982 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:19.982 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:19.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:20.032 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:20.033+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:20.106 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:20.107+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:20.138 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.138 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:20.207 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:20.208+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:17:20.263 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.263 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:20.313 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:20.315+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:20.355 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone36 intact 2024-09-16T22:17:20.409 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:20.410+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:20.425 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.425 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:20.510 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:20.510+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:20.568 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T22:17:20.611 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.611 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.611 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:20.612+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:20.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:20.698 INFO:teuthology.orchestra.run.smithi031.stdout:4074 trunc from 0xaeac13e to 0xb94ec4a 2024-09-16T22:17:20.707 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:20.711 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:20.713+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:20.715 INFO:teuthology.orchestra.run.smithi031.stdout:4076 write 0xd98a6b9 thru 0xd99226b (0x7bb3 bytes) 2024-09-16T22:17:20.724 INFO:teuthology.orchestra.run.smithi031.stdout:4080 write 0xde53ab6 thru 0xde594b8 (0x5a03 bytes) 2024-09-16T22:17:20.727 INFO:teuthology.orchestra.run.smithi031.stdout:4081 write 0x87e4e70 thru 0x87e584f (0x9e0 bytes) 2024-09-16T22:17:20.730 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:20.740 INFO:teuthology.orchestra.run.smithi031.stdout:4085 trunc from 0xde594b9 to 0x4d254d8 2024-09-16T22:17:20.763 INFO:teuthology.orchestra.run.smithi031.stdout:4086 trunc from 0x4d254d8 to 0x51445c1 2024-09-16T22:17:20.781 INFO:teuthology.orchestra.run.smithi031.stdout:4087 write 0xbb1e418 thru 0xbb29c3b (0xb824 bytes) 2024-09-16T22:17:20.782 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.783 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.785 INFO:teuthology.orchestra.run.smithi031.stdout:4088 read 0x7b03a3f thru 0x7b07c98 (0x425a bytes) 2024-09-16T22:17:20.785 INFO:teuthology.orchestra.run.smithi031.stdout:4089 read 0x1ca9560 thru 0x1cb14aa (0x7f4b bytes) 2024-09-16T22:17:20.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:20.797 INFO:teuthology.orchestra.run.smithi031.stdout:4090 trunc from 0xbb29c3c to 0x20c29e9 2024-09-16T22:17:20.812 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:20.813+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:20.891 INFO:teuthology.orchestra.run.smithi031.stdout:4091 trunc from 0x20c29e9 to 0x88bec94 2024-09-16T22:17:20.911 INFO:teuthology.orchestra.run.smithi031.stdout:4093 writesame 0xeb02100 thru 0xeb060a4 data_size 0x1(0x3fa5 bytes) 2024-09-16T22:17:20.911 INFO:teuthology.orchestra.run.smithi031.stdout:4094 write 0x87f2b9b thru 0x87ffd57 (0xd1bd bytes) 2024-09-16T22:17:20.912 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:20.913+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:20.913 INFO:teuthology.orchestra.run.smithi031.stdout:4095 read 0x13e8410 thru 0x13e99ae (0x159f bytes) 2024-09-16T22:17:20.914 INFO:teuthology.orchestra.run.smithi031.stdout:4096 read 0x6e7e9d7 thru 0x6e87378 (0x89a2 bytes) 2024-09-16T22:17:20.915 INFO:teuthology.orchestra.run.smithi031.stdout:4098 writesame 0x779d6ba thru 0x77aa880 data_size 0x99(0xd1c7 bytes) 2024-09-16T22:17:20.921 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:20.921 INFO:teuthology.orchestra.run.smithi031.stdout:4102 read 0x141a5d0 thru 0x1427c53 (0xd684 bytes) 2024-09-16T22:17:20.924 INFO:teuthology.orchestra.run.smithi031.stdout:4103 writesame 0x849522 thru 0x8501d1 data_size 0x5e(0x6cb0 bytes) 2024-09-16T22:17:20.945 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:20.945 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:20.949 INFO:teuthology.orchestra.run.smithi031.stdout:4104 read 0x5eb5a0 thru 0x5f9ed5 (0xe936 bytes) 2024-09-16T22:17:20.954 INFO:teuthology.orchestra.run.smithi031.stdout:4105 read 0x5b42659 thru 0x5b45af2 (0x349a bytes) 2024-09-16T22:17:20.954 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:20.954 INFO:teuthology.orchestra.run.smithi031.stdout:4107 write 0xf452f8 thru 0xf4b7a9 (0x64b2 bytes) 2024-09-16T22:17:20.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:20.972 INFO:teuthology.orchestra.run.smithi031.stdout:4108 write 0x134293b thru 0x134899e (0x6064 bytes) 2024-09-16T22:17:20.987 INFO:teuthology.orchestra.run.smithi031.stdout:4109 read 0x71fc132 thru 0x720b01b (0xeeea bytes) 2024-09-16T22:17:20.987 INFO:teuthology.orchestra.run.smithi031.stdout:4110 writesame 0x5fa1dba thru 0x5fad499 data_size 0x85(0xb6e0 bytes) 2024-09-16T22:17:20.989 INFO:teuthology.orchestra.run.smithi031.stdout:4111 write 0x11a2914 thru 0x11abb26 (0x9213 bytes) 2024-09-16T22:17:21.009 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.009 INFO:teuthology.orchestra.run.smithi031.stdout:4115 read 0x3a6cfeb thru 0x3a73eeb (0x6f01 bytes) 2024-09-16T22:17:21.009 INFO:teuthology.orchestra.run.smithi031.stdout:4118 punch from 0x20bb2ef to 0x20c8861, (0xd572 bytes) 2024-09-16T22:17:21.009 INFO:teuthology.orchestra.run.smithi031.stdout:4120 punch from 0xbb84f08 to 0xbb9113a, (0xc232 bytes) 2024-09-16T22:17:21.013 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:21.015+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:21.021 INFO:teuthology.orchestra.run.smithi031.stdout:4121 trunc from 0xeb060a5 to 0x378a687 2024-09-16T22:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:4122 trunc from 0x378a687 to 0x2b5dff7 2024-09-16T22:17:21.038 INFO:teuthology.orchestra.run.smithi031.stdout:4124 punch from 0xf86cad to 0xf8bf08, (0x525b bytes) 2024-09-16T22:17:21.038 INFO:teuthology.orchestra.run.smithi031.stdout:4125 trunc from 0x2b5dff7 to 0x305ff46 2024-09-16T22:17:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:4127 read 0x1dbc60c thru 0x1dc70c8 (0xaabd bytes) 2024-09-16T22:17:21.046 INFO:teuthology.orchestra.run.smithi031.stdout:4128 punch from 0x2802d80 to 0x2804f6e, (0x21ee bytes) 2024-09-16T22:17:21.046 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.046 INFO:teuthology.orchestra.run.smithi031.stdout:4131 writesame 0xf50de4 thru 0xf5c3f3 data_size 0x10(0xb610 bytes) 2024-09-16T22:17:21.065 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.065 INFO:teuthology.orchestra.run.smithi031.stdout:4133 write 0x109350 thru 0x10b5c4 (0x2275 bytes) 2024-09-16T22:17:21.078 INFO:teuthology.orchestra.run.smithi031.stdout:4134 read 0x10a012c thru 0x10a3359 (0x322e bytes) 2024-09-16T22:17:21.086 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:21.086 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:21.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:21.098 INFO:teuthology.orchestra.run.smithi031.stdout:4135 write 0xcbee5a3 thru 0xcbeff8e (0x19ec bytes) 2024-09-16T22:17:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:4136 trunc from 0xcbeff8f to 0x7469dda 2024-09-16T22:17:21.116 INFO:teuthology.orchestra.run.smithi031.stdout:4137 read 0x63b07c3 thru 0x63bfe29 (0xf667 bytes) 2024-09-16T22:17:21.117 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:21.118+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:21.120 INFO:teuthology.orchestra.run.smithi031.stdout:4138 trunc from 0x7469dda to 0x9f016e3 2024-09-16T22:17:21.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:21.130 INFO:teuthology.orchestra.run.smithi031.stdout:4139 writesame 0xa0bf945 thru 0xa0c1c98 data_size 0x26(0x2354 bytes) 2024-09-16T22:17:21.131 INFO:teuthology.orchestra.run.smithi031.stdout:4141 write 0x3f5be07 thru 0x3f5cf4a (0x1144 bytes) 2024-09-16T22:17:21.143 INFO:teuthology.orchestra.run.smithi031.stdout:4142 writesame 0xce07e6c thru 0xce17740 data_size 0x1(0xf8d5 bytes) 2024-09-16T22:17:21.143 INFO:teuthology.orchestra.run.smithi031.stdout:4143 write 0x56aad4 thru 0x5761ce (0xb6fb bytes) 2024-09-16T22:17:21.169 INFO:teuthology.orchestra.run.smithi031.stdout:4144 write 0x943af7b thru 0x943d96c (0x29f2 bytes) 2024-09-16T22:17:21.172 INFO:teuthology.orchestra.run.smithi031.stdout:4146 read 0x49d40f4 thru 0x49dfb21 (0xba2e bytes) 2024-09-16T22:17:21.172 INFO:teuthology.orchestra.run.smithi031.stdout:4147 punch from 0x38be261 to 0x38c4551, (0x62f0 bytes) 2024-09-16T22:17:21.172 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.172 INFO:teuthology.orchestra.run.smithi031.stdout:4150 read 0x56f4e09 thru 0x56fcf66 (0x815e bytes) 2024-09-16T22:17:21.172 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.178 INFO:teuthology.orchestra.run.smithi031.stdout:4152 trunc from 0xce17741 to 0x77e9063 2024-09-16T22:17:21.198 INFO:teuthology.orchestra.run.smithi031.stdout:4153 write 0xc09b520 thru 0xc09c0bd (0xb9e bytes) 2024-09-16T22:17:21.201 INFO:teuthology.orchestra.run.smithi031.stdout:4155 write 0x431ad97 thru 0x4321b4a (0x6db4 bytes) 2024-09-16T22:17:21.205 INFO:teuthology.orchestra.run.smithi031.stdout:4156 read 0x1cd5ae7 thru 0x1cd91cf (0x36e9 bytes) 2024-09-16T22:17:21.206 INFO:teuthology.orchestra.run.smithi031.stdout:4157 trunc from 0xc09c0be to 0xc955a22 2024-09-16T22:17:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:4159 read 0x862bc58 thru 0x862bd91 (0x13a bytes) 2024-09-16T22:17:21.212 INFO:teuthology.orchestra.run.smithi031.stdout:4162 read 0x3ced19 thru 0x3d534d (0x6635 bytes) 2024-09-16T22:17:21.214 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:21.216+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:21.214 INFO:teuthology.orchestra.run.smithi031.stdout:4163 read 0x19507fd thru 0x1950baf (0x3b3 bytes) 2024-09-16T22:17:21.215 INFO:teuthology.orchestra.run.smithi031.stdout:4164 punch from 0xb294b20 to 0xb29eada, (0x9fba bytes) 2024-09-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:4166 read 0x3e61849 thru 0x3e6f06e (0xd826 bytes) 2024-09-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:4168 punch from 0x91788e0 to 0x9187953, (0xf073 bytes) 2024-09-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:4170 read 0x59344aa thru 0x5939663 (0x51ba bytes) 2024-09-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:4171 read 0x1195c5 thru 0x12425c (0xac98 bytes) 2024-09-16T22:17:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:4172 write 0x280c128 thru 0x280d004 (0xedd bytes) 2024-09-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi031.stdout:4173 read 0x218e2e4 thru 0x219360d (0x532a bytes) 2024-09-16T22:17:21.229 INFO:teuthology.orchestra.run.smithi031.stdout:4174 writesame 0xec0c9bf thru 0xec11953 data_size 0x1(0x4f95 bytes) 2024-09-16T22:17:21.244 INFO:teuthology.orchestra.run.smithi031.stdout:4175 trunc from 0xec11954 to 0x14ef415 2024-09-16T22:17:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:21.259 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:21.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:21.322 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:21.322 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:21.322 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:21.324+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:21.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:21.372 INFO:teuthology.orchestra.run.smithi031.stdout:4176 write 0xd86d231 thru 0xd86e18a (0xf5a bytes) 2024-09-16T22:17:21.378 INFO:teuthology.orchestra.run.smithi031.stdout:4177 trunc from 0xd86e18b to 0xb4377e9 2024-09-16T22:17:21.388 INFO:teuthology.orchestra.run.smithi031.stdout:4179 write 0xb31cbd thru 0xb39778 (0x7abc bytes) 2024-09-16T22:17:21.410 INFO:teuthology.orchestra.run.smithi031.stdout:4182 trunc from 0xb4377e9 to 0x4ee8d8e 2024-09-16T22:17:21.416 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:21.416+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:21.417 INFO:teuthology.orchestra.run.smithi031.stdout:4185 writesame 0x42e1ddc thru 0x42e78e1 data_size 0x3d(0x5b06 bytes) 2024-09-16T22:17:21.433 INFO:teuthology.orchestra.run.smithi031.stdout:4186 write 0xb4f243a thru 0xb4f33b3 (0xf7a bytes) 2024-09-16T22:17:21.436 INFO:teuthology.orchestra.run.smithi031.stdout:4187 punch from 0x583e8a8 to 0x584486a, (0x5fc2 bytes) 2024-09-16T22:17:21.436 INFO:teuthology.orchestra.run.smithi031.stdout:4188 write 0x6ebcd63 thru 0x6ec1a96 (0x4d34 bytes) 2024-09-16T22:17:21.437 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:21.438 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:21.440 INFO:teuthology.orchestra.run.smithi031.stdout:4189 punch from 0x7d169d9 to 0x7d219da, (0xb001 bytes) 2024-09-16T22:17:21.440 INFO:teuthology.orchestra.run.smithi031.stdout:4190 write 0x4b99303 thru 0x4ba3d72 (0xaa70 bytes) 2024-09-16T22:17:21.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:21.453 INFO:teuthology.orchestra.run.smithi031.stdout:4192 writesame 0xe4e6c15 thru 0xe4f365b data_size 0x29(0xca47 bytes) 2024-09-16T22:17:21.453 INFO:teuthology.orchestra.run.smithi031.stdout:4193 write 0x9416c47 thru 0x9419a00 (0x2dba bytes) 2024-09-16T22:17:21.456 INFO:teuthology.orchestra.run.smithi031.stdout:4194 read 0x1832180 thru 0x183f229 (0xd0aa bytes) 2024-09-16T22:17:21.466 INFO:teuthology.orchestra.run.smithi031.stdout:4195 trunc from 0xe4f365c to 0x53566a1 2024-09-16T22:17:21.486 INFO:teuthology.orchestra.run.smithi031.stdout:4196 writesame 0x6359fba thru 0x6363fde data_size 0x1(0xa025 bytes) 2024-09-16T22:17:21.486 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.486 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.498 INFO:teuthology.orchestra.run.smithi031.stdout:4199 write 0xc706a9a thru 0xc714c22 (0xe189 bytes) 2024-09-16T22:17:21.502 INFO:teuthology.orchestra.run.smithi031.stdout:4201 write 0x9007567 thru 0x90092bd (0x1d57 bytes) 2024-09-16T22:17:21.504 INFO:teuthology.orchestra.run.smithi031.stdout:4202 read 0x1e95545 thru 0x1ea0b1b (0xb5d7 bytes) 2024-09-16T22:17:21.505 INFO:teuthology.orchestra.run.smithi031.stdout:4204 write 0xc4a88ee thru 0xc4b7b95 (0xf2a8 bytes) 2024-09-16T22:17:21.511 INFO:teuthology.orchestra.run.smithi031.stdout:4205 read 0x8b0a67c thru 0x8b183e1 (0xdd66 bytes) 2024-09-16T22:17:21.512 INFO:teuthology.orchestra.run.smithi031.stdout:4206 read 0x85ec06d thru 0x85f525e (0x91f2 bytes) 2024-09-16T22:17:21.512 INFO:teuthology.orchestra.run.smithi031.stdout:4207 writesame 0x51502e7 thru 0x5154452 data_size 0x4f(0x416c bytes) 2024-09-16T22:17:21.514 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:21.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:21.514 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T22:17:21.514 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:21.514 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:21.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-16T22:17:21.516 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:21.518+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:21.519 INFO:teuthology.orchestra.run.smithi031.stdout:4208 trunc from 0xc714c23 to 0x7e63925 2024-09-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi031.stdout:4210 trunc from 0x7e63925 to 0x3a60d5 2024-09-16T22:17:21.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:21.597 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:21.597 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:21.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:21.619 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:21.621+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:21.632 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.639 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:21.639 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:4215 writesame 0xb9df0bd thru 0xb9ed8db data_size 0xd(0xe81f bytes) 2024-09-16T22:17:21.650 INFO:teuthology.orchestra.run.smithi031.stdout:4217 punch from 0x8c8f09c to 0x8c997db, (0xa73f bytes) 2024-09-16T22:17:21.651 INFO:teuthology.orchestra.run.smithi031.stdout:4218 trunc from 0xb9ed8dc to 0xc6b0b73 2024-09-16T22:17:21.653 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:21.653 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-16T22:17:21.653 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:4219 read 0x115bf88 thru 0x115ebde (0x2c57 bytes) 2024-09-16T22:17:21.656 INFO:teuthology.orchestra.run.smithi031.stdout:4221 punch from 0x376285c to 0x37720fb, (0xf89f bytes) 2024-09-16T22:17:21.657 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.657 INFO:teuthology.orchestra.run.smithi031.stdout:4227 write 0x521a548 thru 0x5222301 (0x7dba bytes) 2024-09-16T22:17:21.658 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:21.658 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:21.661 INFO:teuthology.orchestra.run.smithi031.stdout:4228 trunc from 0xc6b0b73 to 0xd0afbd1 2024-09-16T22:17:21.668 INFO:teuthology.orchestra.run.smithi031.stdout:4230 read 0xbc0767b thru 0xbc110d7 (0x9a5d bytes) 2024-09-16T22:17:21.668 INFO:teuthology.orchestra.run.smithi031.stdout:4232 read 0x346ac94 thru 0x34704c0 (0x582d bytes) 2024-09-16T22:17:21.668 INFO:teuthology.orchestra.run.smithi031.stdout:4233 punch from 0x49412a1 to 0x4948d6e, (0x7acd bytes) 2024-09-16T22:17:21.668 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.669 INFO:teuthology.orchestra.run.smithi031.stdout:4236 trunc from 0xd0afbd1 to 0xb120df0 2024-09-16T22:17:21.676 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:21.680 INFO:teuthology.orchestra.run.smithi031.stdout:4237 write 0xb62aaf9 thru 0xb62c1d9 (0x16e1 bytes) 2024-09-16T22:17:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:4238 read 0xaecbc1 thru 0xaf4f9f (0x83df bytes) 2024-09-16T22:17:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.683 INFO:teuthology.orchestra.run.smithi031.stdout:4245 write 0x27fa77f thru 0x2804f1d (0xa79f bytes) 2024-09-16T22:17:21.687 INFO:teuthology.orchestra.run.smithi031.stdout:4248 read 0x465820b thru 0x4661e8c (0x9c82 bytes) 2024-09-16T22:17:21.687 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.687 INFO:teuthology.orchestra.run.smithi031.stdout:4250 write 0x940a7c3 thru 0x94132cc (0x8b0a bytes) 2024-09-16T22:17:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:4251 write 0xb2fb5a7 thru 0xb307e10 (0xc86a bytes) 2024-09-16T22:17:21.694 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:21.709 INFO:teuthology.orchestra.run.smithi031.stdout:4252 trunc from 0xb62c1da to 0x161da12 2024-09-16T22:17:21.718 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:21.720+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:21.720 INFO:teuthology.orchestra.run.smithi031.stdout:4255 punch from 0x125af64 to 0x125f93d, (0x49d9 bytes) 2024-09-16T22:17:21.720 INFO:teuthology.orchestra.run.smithi031.stdout:4256 read 0x3c0365 thru 0x3c510b (0x4da7 bytes) 2024-09-16T22:17:21.726 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:21.726 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:21.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:21.738 INFO:teuthology.orchestra.run.smithi031.stdout:4257 write 0xb2e9d01 thru 0xb2f46c7 (0xa9c7 bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4258 punch from 0x12ab48d to 0x12b178f, (0x6302 bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4259 punch from 0xa0ad6ca to 0xa0b8827, (0xb15d bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4260 punch from 0x31be803 to 0x31c6f07, (0x8704 bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4261 punch from 0xb23ae68 to 0xb23f565, (0x46fd bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4267 read 0x4bfd741 thru 0x4c0ae0f (0xd6cf bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4268 read 0x9af7e29 thru 0x9afcd94 (0x4f6c bytes) 2024-09-16T22:17:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:4269 write 0x4ea0471 thru 0x4ea616b (0x5cfb bytes) 2024-09-16T22:17:21.749 INFO:teuthology.orchestra.run.smithi031.stdout:4270 write 0x418b9d thru 0x41b515 (0x2979 bytes) 2024-09-16T22:17:21.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.751 INFO:teuthology.orchestra.run.smithi031.stdout:4272 trunc from 0xb2f46c8 to 0xa91ed8c 2024-09-16T22:17:21.764 INFO:teuthology.orchestra.run.smithi031.stdout:4277 read 0x4839333 thru 0x4846bd4 (0xd8a2 bytes) 2024-09-16T22:17:21.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.771 INFO:teuthology.orchestra.run.smithi031.stdout:4280 trunc from 0xa91ed8c to 0x45d0ad1 2024-09-16T22:17:21.785 INFO:teuthology.orchestra.run.smithi031.stdout:4281 trunc from 0x45d0ad1 to 0x7deb196 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4283 writesame 0x9520f95 thru 0x952334c data_size 0x24(0x23b8 bytes) 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4284 read 0x6633292 thru 0x663fc9c (0xca0b bytes) 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4285 read 0x8b88a70 thru 0x8b8e59a (0x5b2b bytes) 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4286 punch from 0x12bd567 to 0x12c4938, (0x73d1 bytes) 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4289 read 0x35da88d thru 0x35dad7f (0x4f3 bytes) 2024-09-16T22:17:21.798 INFO:teuthology.orchestra.run.smithi031.stdout:4290 write 0x78b5ce2 thru 0x78bdd2f (0x804e bytes) 2024-09-16T22:17:21.800 INFO:teuthology.orchestra.run.smithi031.stdout:4291 punch from 0x136ce1d to 0x1378a31, (0xbc14 bytes) 2024-09-16T22:17:21.806 INFO:teuthology.orchestra.run.smithi031.stdout:4293 trunc from 0x952334d to 0xe2466f8 2024-09-16T22:17:21.811 INFO:teuthology.orchestra.run.smithi031.stdout:4294 writesame 0x78016bc thru 0x780985b data_size 0x44(0x81a0 bytes) 2024-09-16T22:17:21.815 INFO:teuthology.orchestra.run.smithi031.stdout:4296 write 0xc66a43a thru 0xc6767ab (0xc372 bytes) 2024-09-16T22:17:21.819 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:21.820+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:21.828 INFO:teuthology.orchestra.run.smithi031.stdout:4297 writesame 0xecbead4 thru 0xeccb8e3 data_size 0x9d(0xce10 bytes) 2024-09-16T22:17:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:4298 writesame 0x2a1230d thru 0x2a14a46 data_size 0x1(0x273a bytes) 2024-09-16T22:17:21.835 INFO:teuthology.orchestra.run.smithi031.stdout:4299 read 0xd3f57ee thru 0xd3ff0ec (0x98ff bytes) 2024-09-16T22:17:21.835 INFO:teuthology.orchestra.run.smithi031.stdout:4300 write 0xa934a36 thru 0xa94392a (0xeef5 bytes) 2024-09-16T22:17:21.841 INFO:teuthology.orchestra.run.smithi031.stdout:4302 punch from 0xd504485 to 0xd510400, (0xbf7b bytes) 2024-09-16T22:17:21.841 INFO:teuthology.orchestra.run.smithi031.stdout:4304 write 0x4f23a4c thru 0x4f31676 (0xdc2b bytes) 2024-09-16T22:17:21.847 INFO:teuthology.orchestra.run.smithi031.stdout:4305 read 0x7ac6197 thru 0x7ac9808 (0x3672 bytes) 2024-09-16T22:17:21.847 INFO:teuthology.orchestra.run.smithi031.stdout:4306 write 0xe44e1fd thru 0xe456f2c (0x8d30 bytes) 2024-09-16T22:17:21.847 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:21.847 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:21.848 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:21.848 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:21.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:21.862 INFO:teuthology.orchestra.run.smithi031.stdout:4307 trunc from 0xeccb8e4 to 0x657027a 2024-09-16T22:17:21.875 INFO:teuthology.orchestra.run.smithi031.stdout:4308 read 0x55f3bbd thru 0x55f715b (0x359f bytes) 2024-09-16T22:17:21.875 INFO:teuthology.orchestra.run.smithi031.stdout:4312 punch from 0x488bdbf to 0x489b005, (0xf246 bytes) 2024-09-16T22:17:21.875 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.876 INFO:teuthology.orchestra.run.smithi031.stdout:4315 punch from 0x51020d8 to 0x5105bbd, (0x3ae5 bytes) 2024-09-16T22:17:21.884 INFO:teuthology.orchestra.run.smithi031.stdout:4316 write 0x8ed6455 thru 0x8edaa8e (0x463a bytes) 2024-09-16T22:17:21.887 INFO:teuthology.orchestra.run.smithi031.stdout:4317 write 0x8243aff thru 0x82455be (0x1ac0 bytes) 2024-09-16T22:17:21.890 INFO:teuthology.orchestra.run.smithi031.stdout:4318 write 0x5289cc7 thru 0x5295d7f (0xc0b9 bytes) 2024-09-16T22:17:21.893 INFO:teuthology.orchestra.run.smithi031.stdout:4319 read 0x1e017ab thru 0x1e01f50 (0x7a6 bytes) 2024-09-16T22:17:21.894 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:21.894 INFO:teuthology.orchestra.run.smithi031.stdout:4321 trunc from 0x8edaa8f to 0x966c474 2024-09-16T22:17:21.899 INFO:teuthology.orchestra.run.smithi031.stdout:4322 write 0x776b18 thru 0x782de4 (0xc2cd bytes) 2024-09-16T22:17:21.913 INFO:teuthology.orchestra.run.smithi031.stdout:4323 trunc from 0x966c474 to 0x1f36876 2024-09-16T22:17:21.919 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:21.921+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:21.925 INFO:teuthology.orchestra.run.smithi031.stdout:4324 read 0x1069828 thru 0x106af67 (0x1740 bytes) 2024-09-16T22:17:21.934 INFO:teuthology.orchestra.run.smithi031.stdout:4325 write 0x3a6f692 thru 0x3a76279 (0x6be8 bytes) 2024-09-16T22:17:21.937 INFO:teuthology.orchestra.run.smithi031.stdout:4326 read 0xe62ff1 thru 0xe63261 (0x271 bytes) 2024-09-16T22:17:21.954 INFO:teuthology.orchestra.run.smithi031.stdout:4327 writesame 0xdce2971 thru 0xdcec1ac data_size 0x2(0x983c bytes) 2024-09-16T22:17:21.954 INFO:teuthology.orchestra.run.smithi031.stdout:4328 write 0x9e1d8f9 thru 0x9e253cc (0x7ad4 bytes) 2024-09-16T22:17:21.963 INFO:teuthology.orchestra.run.smithi031.stdout:4329 write 0xdd57fcc thru 0xdd61fd2 (0xa007 bytes) 2024-09-16T22:17:21.972 INFO:teuthology.orchestra.run.smithi031.stdout:4330 trunc from 0xdd61fd3 to 0x9298df0 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3830903769999998, 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3830518029999999, 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3830321539999999, 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382990741, 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3829548909999998, 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3829369680000001, 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382917387, 2024-09-16T22:17:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382891876, 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382873494, 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3828549539999999, 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3828367739999998, 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3828192079999999, 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382797246, 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3827793220000002, 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382761812, 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3827441409999999, 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3823021999999998, 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.382186903, 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3821599039999999, 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3821065269999999, 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:22.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.068 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:22.070 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:22.069+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:22.071 INFO:teuthology.orchestra.run.smithi031.stdout:4332 read 0x4af252a thru 0x4af3a15 (0x14ec bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4333 writesame 0x397a125 thru 0x3984509 data_size 0x1(0xa3e5 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4334 punch from 0xd1a8cd to 0xd2a24c, (0xf97f bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4335 write 0x30f557c thru 0x30ff87c (0xa301 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4337 read 0x1418039 thru 0x141a90b (0x28d3 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4338 punch from 0x5afda50 to 0x5b0272c, (0x4cdc bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4339 writesame 0xdc8a905 thru 0xdc99e8a data_size 0xb(0xf586 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4340 read 0x21005f9 thru 0x210d7e9 (0xd1f1 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4343 punch from 0x61a4c42 to 0x61adfb8, (0x9376 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4345 trunc from 0xdc99e8b to 0x1bd5d4c 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4346 read 0x1697b4f thru 0x169f890 (0x7d42 bytes) 2024-09-16T22:17:22.072 INFO:teuthology.orchestra.run.smithi031.stdout:4347 trunc from 0x1bd5d4c to 0x1ed135d 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4350 writesame 0x9ad4d4e thru 0x9ad591b data_size 0x1(0xbce bytes) 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4351 read 0xc0373c thru 0xc06ecd (0x3792 bytes) 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4352 writesame 0xd4f467b thru 0xd4fc472 data_size 0x1d(0x7df8 bytes) 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4355 read 0x5360d53 thru 0x5366014 (0x52c2 bytes) 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4356 write 0xd76a684 thru 0xd77449f (0x9e1c bytes) 2024-09-16T22:17:22.073 INFO:teuthology.orchestra.run.smithi031.stdout:4357 writesame 0x7a3a1e2 thru 0x7a48938 data_size 0x13(0xe757 bytes) 2024-09-16T22:17:22.079 INFO:teuthology.orchestra.run.smithi031.stdout:4358 write 0xd139487 thru 0xd1450e4 (0xbc5e bytes) 2024-09-16T22:17:22.085 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.086 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.086 INFO:teuthology.orchestra.run.smithi031.stdout:4362 punch from 0x18333f2 to 0x183df9a, (0xaba8 bytes) 2024-09-16T22:17:22.086 INFO:teuthology.orchestra.run.smithi031.stdout:4364 punch from 0x5421a27 to 0x54292a4, (0x787d bytes) 2024-09-16T22:17:22.086 INFO:teuthology.orchestra.run.smithi031.stdout:4365 read 0x16c7596 thru 0x16d6bbc (0xf627 bytes) 2024-09-16T22:17:22.092 INFO:teuthology.orchestra.run.smithi031.stdout:4366 write 0xe6b13cb thru 0xe6b3ac6 (0x26fc bytes) 2024-09-16T22:17:22.095 INFO:teuthology.orchestra.run.smithi031.stdout:4369 read 0x4ecab88 thru 0x4eda146 (0xf5bf bytes) 2024-09-16T22:17:22.095 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.095 INFO:teuthology.orchestra.run.smithi031.stdout:4371 write 0x90bfceb thru 0x90ca6fb (0xaa11 bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4372 punch from 0xa0002e to 0xa014a3, (0x1475 bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4373 read 0xe028a5 thru 0xe084fa (0x5c56 bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4374 read 0x9c3792c thru 0x9c38dca (0x149f bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4375 read 0x6ba3848 thru 0x6badc4d (0xa406 bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4376 punch from 0xec8310 to 0xed43b4, (0xc0a4 bytes) 2024-09-16T22:17:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:4378 write 0x66fc012 thru 0x66ffb9d (0x3b8c bytes) 2024-09-16T22:17:22.101 INFO:teuthology.orchestra.run.smithi031.stdout:4380 write 0x19998d8 thru 0x199d007 (0x3730 bytes) 2024-09-16T22:17:22.104 INFO:teuthology.orchestra.run.smithi031.stdout:4381 writesame 0x29f3132 thru 0x2a010a7 data_size 0x1(0xdf76 bytes) 2024-09-16T22:17:22.133 INFO:teuthology.orchestra.run.smithi031.stdout:4382 read 0x796f8ee thru 0x797a611 (0xad24 bytes) 2024-09-16T22:17:22.134 INFO:teuthology.orchestra.run.smithi031.stdout:4383 write 0x4652cb3 thru 0x465f2d9 (0xc627 bytes) 2024-09-16T22:17:22.137 INFO:teuthology.orchestra.run.smithi031.stdout:4384 read 0xb6724bf thru 0xb677eaa (0x59ec bytes) 2024-09-16T22:17:22.137 INFO:teuthology.orchestra.run.smithi031.stdout:4385 write 0x919a8ae thru 0x919af34 (0x687 bytes) 2024-09-16T22:17:22.140 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.140 INFO:teuthology.orchestra.run.smithi031.stdout:4387 read 0xe5bb0e5 thru 0xe5bb81a (0x736 bytes) 2024-09-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi031.stdout:4388 trunc from 0xe6b3ac7 to 0x67f9074 2024-09-16T22:17:22.162 INFO:teuthology.orchestra.run.smithi031.stdout:4389 punch from 0x5c37cf6 to 0x5c37e9f, (0x1a9 bytes) 2024-09-16T22:17:22.166 INFO:teuthology.orchestra.run.smithi031.stdout:4390 trunc from 0x67f9074 to 0x2df2231 2024-09-16T22:17:22.170 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:22.171+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:22.181 INFO:teuthology.orchestra.run.smithi031.stdout:4393 read 0x18fe8d5 thru 0x190cf8e (0xe6ba bytes) 2024-09-16T22:17:22.181 INFO:teuthology.orchestra.run.smithi031.stdout:4394 trunc from 0x2df2231 to 0x295a2e8 2024-09-16T22:17:22.201 INFO:teuthology.orchestra.run.smithi031.stdout:4395 write 0x4957a2c thru 0x4959418 (0x19ed bytes) 2024-09-16T22:17:22.204 INFO:teuthology.orchestra.run.smithi031.stdout:4396 punch from 0xc0c890 to 0xc192ee, (0xca5e bytes) 2024-09-16T22:17:22.205 INFO:teuthology.orchestra.run.smithi031.stdout:4398 write 0x3d54502 thru 0x3d5769c (0x319b bytes) 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:4399 punch from 0x28cbcc2 to 0x28cdbc6, (0x1f04 bytes) 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:4400 punch from 0x364e354 to 0x365bcf5, (0xd9a1 bytes) 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:4401 read 0x2849820 thru 0x284c38a (0x2b6b bytes) 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:4406 read 0x3fbf55f thru 0x3fcbdc3 (0xc865 bytes) 2024-09-16T22:17:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.209 INFO:teuthology.orchestra.run.smithi031.stdout:4408 read 0x3bb411f thru 0x3bc1384 (0xd266 bytes) 2024-09-16T22:17:22.210 INFO:teuthology.orchestra.run.smithi031.stdout:4410 trunc from 0x4959419 to 0x6978f6f 2024-09-16T22:17:22.223 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:22.223 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:22.223 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:22.223 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.226 INFO:teuthology.orchestra.run.smithi031.stdout:4411 write 0x9498ca6 thru 0x949f6c1 (0x6a1c bytes) 2024-09-16T22:17:22.229 INFO:teuthology.orchestra.run.smithi031.stdout:4412 read 0x8730e43 thru 0x873e0bb (0xd279 bytes) 2024-09-16T22:17:22.229 INFO:teuthology.orchestra.run.smithi031.stdout:4413 write 0x3099387 thru 0x30a5960 (0xc5da bytes) 2024-09-16T22:17:22.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:22.246 INFO:teuthology.orchestra.run.smithi031.stdout:4414 write 0xe5e3d96 thru 0xe5e7d18 (0x3f83 bytes) 2024-09-16T22:17:22.249 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.249 INFO:teuthology.orchestra.run.smithi031.stdout:4417 write 0x74e837e thru 0x74f53d1 (0xd054 bytes) 2024-09-16T22:17:22.256 INFO:teuthology.orchestra.run.smithi031.stdout:4418 punch from 0xd0c0e60 to 0xd0c465d, (0x37fd bytes) 2024-09-16T22:17:22.257 INFO:teuthology.orchestra.run.smithi031.stdout:4419 write 0x63e5806 thru 0x63e7341 (0x1b3c bytes) 2024-09-16T22:17:22.259 INFO:teuthology.orchestra.run.smithi031.stdout:4420 write 0x4b8a32a thru 0x4b93a1a (0x96f1 bytes) 2024-09-16T22:17:22.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:17:22.264 INFO:teuthology.orchestra.run.smithi031.stdout:4422 write 0x14f7d4c thru 0x14fb0bd (0x3372 bytes) 2024-09-16T22:17:22.267 INFO:teuthology.orchestra.run.smithi031.stdout:4423 write 0xb8a83a4 thru 0xb8b689a (0xe4f7 bytes) 2024-09-16T22:17:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:4426 write 0x902bea8 thru 0x9033b07 (0x7c60 bytes) 2024-09-16T22:17:22.274 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:22.275+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:22.280 INFO:teuthology.orchestra.run.smithi031.stdout:4427 read 0x1943d1c thru 0x19500a6 (0xc38b bytes) 2024-09-16T22:17:22.280 INFO:teuthology.orchestra.run.smithi031.stdout:4428 write 0xc4e45ea thru 0xc4edf71 (0x9988 bytes) 2024-09-16T22:17:22.284 INFO:teuthology.orchestra.run.smithi031.stdout:4429 punch from 0x29223f2 to 0x292ff21, (0xdb2f bytes) 2024-09-16T22:17:22.284 INFO:teuthology.orchestra.run.smithi031.stdout:4430 writesame 0x560375b thru 0x560ad0f data_size 0x1(0x75b5 bytes) 2024-09-16T22:17:22.302 INFO:teuthology.orchestra.run.smithi031.stdout:4432 trunc from 0xe5e7d19 to 0xd8b684e 2024-09-16T22:17:22.322 INFO:teuthology.orchestra.run.smithi031.stdout:4433 trunc from 0xd8b684e to 0x7148788 2024-09-16T22:17:22.340 INFO:teuthology.orchestra.run.smithi031.stdout:4434 trunc from 0x7148788 to 0x12505da 2024-09-16T22:17:22.357 INFO:teuthology.orchestra.run.smithi031.stdout:4435 read 0x6c1865 thru 0x6cd2e7 (0xba83 bytes) 2024-09-16T22:17:22.357 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.368 INFO:teuthology.orchestra.run.smithi031.stdout:4437 writesame 0x527fa3e thru 0x52885ab data_size 0x1b(0x8b6e bytes) 2024-09-16T22:17:22.372 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:22.373+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:22.375 INFO:teuthology.orchestra.run.smithi031.stdout:4438 write 0x6448c9a thru 0x6455460 (0xc7c7 bytes) 2024-09-16T22:17:22.380 INFO:teuthology.orchestra.run.smithi031.stdout:4441 read 0x54d9161 thru 0x54d9885 (0x725 bytes) 2024-09-16T22:17:22.380 INFO:teuthology.orchestra.run.smithi031.stdout:4443 punch from 0x9d2bc4 to 0x9e23b0, (0xf7ec bytes) 2024-09-16T22:17:22.381 INFO:teuthology.orchestra.run.smithi031.stdout:4444 read 0x5bcc8c9 thru 0x5bd37d0 (0x6f08 bytes) 2024-09-16T22:17:22.381 INFO:teuthology.orchestra.run.smithi031.stdout:4445 write 0x529121f thru 0x52912e8 (0xca bytes) 2024-09-16T22:17:22.383 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:22.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:22.383 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:22.384 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.384 INFO:teuthology.orchestra.run.smithi031.stdout:4446 punch from 0x16a6e6a to 0x16b237c, (0xb512 bytes) 2024-09-16T22:17:22.384 INFO:teuthology.orchestra.run.smithi031.stdout:4449 read 0x574959a thru 0x5757e91 (0xe8f8 bytes) 2024-09-16T22:17:22.391 INFO:teuthology.orchestra.run.smithi031.stdout:4450 trunc from 0x6455461 to 0xbed85fa 2024-09-16T22:17:22.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:22.410 INFO:teuthology.orchestra.run.smithi031.stdout:4451 trunc from 0xbed85fa to 0x8fe571 2024-09-16T22:17:22.421 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.421 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.421 INFO:teuthology.orchestra.run.smithi031.stdout:4454 read 0x22ca6e thru 0x230252 (0x37e5 bytes) 2024-09-16T22:17:22.422 INFO:teuthology.orchestra.run.smithi031.stdout:4455 trunc from 0x8fe571 to 0x16e92dc 2024-09-16T22:17:22.428 INFO:teuthology.orchestra.run.smithi031.stdout:4456 read 0x11b7096 thru 0x11c0795 (0x9700 bytes) 2024-09-16T22:17:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:4458 writesame 0xc613ecf thru 0xc61f40e data_size 0x5(0xb540 bytes) 2024-09-16T22:17:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:4461 read 0x7168900 thru 0x716b4a6 (0x2ba7 bytes) 2024-09-16T22:17:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:4463 read 0xa6fe056 thru 0xa6ff62f (0x15da bytes) 2024-09-16T22:17:22.457 INFO:teuthology.orchestra.run.smithi031.stdout:4464 trunc from 0xc61f40f to 0x25395ae 2024-09-16T22:17:22.463 INFO:teuthology.orchestra.run.smithi031.stdout:4466 writesame 0x16d0d28 thru 0x16d7fc4 data_size 0x25(0x729d bytes) 2024-09-16T22:17:22.466 INFO:teuthology.orchestra.run.smithi031.stdout:4467 read 0xd30ec9 thru 0xd39acd (0x8c05 bytes) 2024-09-16T22:17:22.473 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:22.475+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:22.475 INFO:teuthology.orchestra.run.smithi031.stdout:4470 write 0x5010048 thru 0x50155a1 (0x555a bytes) 2024-09-16T22:17:22.493 INFO:teuthology.orchestra.run.smithi031.stdout:4471 writesame 0xd80bce2 thru 0xd80d9b6 data_size 0xb(0x1cd5 bytes) 2024-09-16T22:17:22.493 INFO:teuthology.orchestra.run.smithi031.stdout:4472 write 0x19066cf thru 0x190e66e (0x7fa0 bytes) 2024-09-16T22:17:22.497 INFO:teuthology.orchestra.run.smithi031.stdout:4473 punch from 0x47552c7 to 0x4760280, (0xafb9 bytes) 2024-09-16T22:17:22.497 INFO:teuthology.orchestra.run.smithi031.stdout:4475 read 0x1cb1b6 thru 0x1d6126 (0xaf71 bytes) 2024-09-16T22:17:22.497 INFO:teuthology.orchestra.run.smithi031.stdout:4477 writesame 0xd3aa313 thru 0xd3acfee data_size 0x57(0x2cdc bytes) 2024-09-16T22:17:22.501 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.996764469, 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995218692, 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995200612, 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995184050000001, 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995167562000001, 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.99515049, 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:22.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995133984000001, 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.533 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.995104191999999, 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:22.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:22.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990771935, 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:22.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:22.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:22.542 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990749674, 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:22.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:22.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.99072868, 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:22.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990707931999999, 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:22.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:22.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.99068763, 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:22.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990667627000001, 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990647502, 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:22.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990627907, 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:22.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:22.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990606984999999, 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:22.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990586197000001, 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:22.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990564536000001, 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:22.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:22.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.990535445999999, 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:22.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:22.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:22.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:22.592 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:22.594 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:22.594+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4478 trunc from 0xd80d9b7 to 0xc3f5621 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4480 trunc from 0xc3f5621 to 0xdb21324 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4483 read 0x10c1809 thru 0x10cd3a6 (0xbb9e bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4484 read 0x2da07c1 thru 0x2da3004 (0x2844 bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4485 trunc from 0xdb21324 to 0x5e618bd 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4486 write 0x3063d56 thru 0x3066d70 (0x301b bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4487 trunc from 0x5e618bd to 0x79c362e 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4488 read 0x2b07d0f thru 0x2b13cca (0xbfbc bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4489 write 0xafc7e5f thru 0xafd7543 (0xf6e5 bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4490 trunc from 0xafd7544 to 0xb4e7054 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4492 write 0x78c9ba2 thru 0x78ca81b (0xc7a bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4493 read 0x5cd2f91 thru 0x5cd4030 (0x10a0 bytes) 2024-09-16T22:17:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:4494 punch from 0xa45482a to 0xa45663c, (0x1e12 bytes) 2024-09-16T22:17:22.597 INFO:teuthology.orchestra.run.smithi031.stdout:4495 trunc from 0xb4e7054 to 0x88dadf8 2024-09-16T22:17:22.597 INFO:teuthology.orchestra.run.smithi031.stdout:4496 write 0xa748128 thru 0xa749837 (0x1710 bytes) 2024-09-16T22:17:22.597 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.597 INFO:teuthology.orchestra.run.smithi031.stdout:4498 writesame 0x5ed5736 thru 0x5ee05d0 data_size 0x1(0xae9b bytes) 2024-09-16T22:17:22.617 INFO:teuthology.orchestra.run.smithi031.stdout:4499 punch from 0x3321f5a to 0x332fb5e, (0xdc04 bytes) 2024-09-16T22:17:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:4500 write 0xc467d48 thru 0xc47333f (0xb5f8 bytes) 2024-09-16T22:17:22.629 INFO:teuthology.orchestra.run.smithi031.stdout:4501 flatten 2024-09-16T22:17:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:4502 punch from 0xb36b899 to 0xb3705d1, (0x4d38 bytes) 2024-09-16T22:17:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:4505 trunc from 0xc473340 to 0xbde80ef 2024-09-16T22:17:22.649 INFO:teuthology.orchestra.run.smithi031.stdout:4506 read 0x3552f98 thru 0x355f8f8 (0xc961 bytes) 2024-09-16T22:17:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:4507 read 0x34096db thru 0x340c249 (0x2b6f bytes) 2024-09-16T22:17:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:4509 punch from 0x8c3889a to 0x8c4404d, (0xb7b3 bytes) 2024-09-16T22:17:22.656 INFO:teuthology.orchestra.run.smithi031.stdout:4511 writesame 0xcdc34a1 thru 0xcdc5ab5 data_size 0x1(0x2615 bytes) 2024-09-16T22:17:22.656 INFO:teuthology.orchestra.run.smithi031.stdout:4512 read 0xb81f419 thru 0xb829375 (0x9f5d bytes) 2024-09-16T22:17:22.665 INFO:teuthology.orchestra.run.smithi031.stdout:4514 trunc from 0xcdc5ab6 to 0x52fdc8e 2024-09-16T22:17:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:4515 punch from 0x51398d6 to 0x513ec3f, (0x5369 bytes) 2024-09-16T22:17:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:4516 read 0x1575bb7 thru 0x1580ef8 (0xb342 bytes) 2024-09-16T22:17:22.676 INFO:teuthology.orchestra.run.smithi031.stdout:4517 write 0x1fc4d67 thru 0x1fca9e1 (0x5c7b bytes) 2024-09-16T22:17:22.693 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:22.694+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:22.694 INFO:teuthology.orchestra.run.smithi031.stdout:4518 write 0xcedc4d3 thru 0xcee36cd (0x71fb bytes) 2024-09-16T22:17:22.697 INFO:teuthology.orchestra.run.smithi031.stdout:4519 read 0x9c2e792 thru 0x9c3b0ad (0xc91c bytes) 2024-09-16T22:17:22.698 INFO:teuthology.orchestra.run.smithi031.stdout:4520 writesame 0x25fc0ec thru 0x25ff9e6 data_size 0x1(0x38fb bytes) 2024-09-16T22:17:22.706 INFO:teuthology.orchestra.run.smithi031.stdout:4521 read 0x8e510cc thru 0x8e5ef8a (0xdebf bytes) 2024-09-16T22:17:22.706 INFO:teuthology.orchestra.run.smithi031.stdout:4523 write 0x1c6b8bf thru 0x1c703a7 (0x4ae9 bytes) 2024-09-16T22:17:22.710 INFO:teuthology.orchestra.run.smithi031.stdout:4525 writesame 0x736045b thru 0x736c941 data_size 0x13(0xc4e7 bytes) 2024-09-16T22:17:22.714 INFO:teuthology.orchestra.run.smithi031.stdout:4526 write 0x9fae202 thru 0x9fb5abb (0x78ba bytes) 2024-09-16T22:17:22.718 INFO:teuthology.orchestra.run.smithi031.stdout:4527 write 0xab5b4c0 thru 0xab63054 (0x7b95 bytes) 2024-09-16T22:17:22.722 INFO:teuthology.orchestra.run.smithi031.stdout:4529 read 0x551e345 thru 0x552a8a1 (0xc55d bytes) 2024-09-16T22:17:22.722 INFO:teuthology.orchestra.run.smithi031.stdout:4531 read 0x3ccdc7c thru 0x3cd2c79 (0x4ffe bytes) 2024-09-16T22:17:22.722 INFO:teuthology.orchestra.run.smithi031.stdout:4532 write 0x96ee01e thru 0x96ef2c4 (0x12a7 bytes) 2024-09-16T22:17:22.733 INFO:teuthology.orchestra.run.smithi031.stdout:4533 writesame 0xeb02e8f thru 0xeb0ef09 data_size 0x4b(0xc07b bytes) 2024-09-16T22:17:22.733 INFO:teuthology.orchestra.run.smithi031.stdout:4534 punch from 0x1a1f4ae to 0x1a2a00b, (0xab5d bytes) 2024-09-16T22:17:22.733 INFO:teuthology.orchestra.run.smithi031.stdout:4535 read 0x9e2ef51 thru 0x9e34be9 (0x5c99 bytes) 2024-09-16T22:17:22.734 INFO:teuthology.orchestra.run.smithi031.stdout:4536 read 0x39866b5 thru 0x398b082 (0x49ce bytes) 2024-09-16T22:17:22.734 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.734 INFO:teuthology.orchestra.run.smithi031.stdout:4538 write 0xc6ac6e thru 0xc71753 (0x6ae6 bytes) 2024-09-16T22:17:22.743 INFO:teuthology.orchestra.run.smithi031.stdout:4539 trunc from 0xeb0ef0a to 0x8efda15 2024-09-16T22:17:22.747 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:22.747 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:22.747 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:22.747 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:22.760 INFO:teuthology.orchestra.run.smithi031.stdout:4540 trunc from 0x8efda15 to 0xcad21b6 2024-09-16T22:17:22.766 INFO:teuthology.orchestra.run.smithi031.stdout:4541 read 0x93d5f57 thru 0x93e26cc (0xc776 bytes) 2024-09-16T22:17:22.766 INFO:teuthology.orchestra.run.smithi031.stdout:4542 write 0x12cd6d4 thru 0x12daf59 (0xd886 bytes) 2024-09-16T22:17:22.773 INFO:teuthology.orchestra.run.smithi031.stdout:4543 read 0x298a27 thru 0x29e119 (0x56f3 bytes) 2024-09-16T22:17:22.774 INFO:teuthology.orchestra.run.smithi031.stdout:4544 punch from 0x13ffa0 to 0x14402f, (0x408f bytes) 2024-09-16T22:17:22.780 INFO:teuthology.orchestra.run.smithi031.stdout:4545 write 0xcdff382 thru 0xcdff4c0 (0x13f bytes) 2024-09-16T22:17:22.783 INFO:teuthology.orchestra.run.smithi031.stdout:4546 read 0x97e5851 thru 0x97f4f60 (0xf710 bytes) 2024-09-16T22:17:22.783 INFO:teuthology.orchestra.run.smithi031.stdout:4548 write 0xaa492bd thru 0xaa57378 (0xe0bc bytes) 2024-09-16T22:17:22.787 INFO:teuthology.orchestra.run.smithi031.stdout:4549 read 0x7353ffa thru 0x7361920 (0xd927 bytes) 2024-09-16T22:17:22.787 INFO:teuthology.orchestra.run.smithi031.stdout:4550 writesame 0x75eb56c thru 0x75f4b4a data_size 0x93(0x95df bytes) 2024-09-16T22:17:22.792 INFO:teuthology.orchestra.run.smithi031.stdout:4551 punch from 0x2b0926a to 0x2b122bb, (0x9051 bytes) 2024-09-16T22:17:22.792 INFO:teuthology.orchestra.run.smithi031.stdout:4552 write 0x7b5e61d thru 0x7b61cbe (0x36a2 bytes) 2024-09-16T22:17:22.797 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:22.798+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:22.799 INFO:teuthology.orchestra.run.smithi031.stdout:4553 read 0x35e4adb thru 0x35ed13b (0x8661 bytes) 2024-09-16T22:17:22.799 INFO:teuthology.orchestra.run.smithi031.stdout:4556 punch from 0x30a9cb3 to 0x30ab50d, (0x185a bytes) 2024-09-16T22:17:22.799 INFO:teuthology.orchestra.run.smithi031.stdout:4557 punch from 0x35be9a0 to 0x35cd200, (0xe860 bytes) 2024-09-16T22:17:22.805 INFO:teuthology.orchestra.run.smithi031.stdout:4558 write 0xd917dc3 thru 0xd91a634 (0x2872 bytes) 2024-09-16T22:17:22.809 INFO:teuthology.orchestra.run.smithi031.stdout:4559 writesame 0x7dbb006 thru 0x7dc4c65 data_size 0x8b(0x9c60 bytes) 2024-09-16T22:17:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:4560 read 0x9779e8 thru 0x9855b8 (0xdbd1 bytes) 2024-09-16T22:17:22.829 INFO:teuthology.orchestra.run.smithi031.stdout:4561 trunc from 0xd91a635 to 0x17e091b 2024-09-16T22:17:22.842 INFO:teuthology.orchestra.run.smithi031.stdout:4562 write 0x113e6d1 thru 0x114d92a (0xf25a bytes) 2024-09-16T22:17:22.865 INFO:teuthology.orchestra.run.smithi031.stdout:4563 write 0xa70d78a thru 0xa71a7d3 (0xd04a bytes) 2024-09-16T22:17:22.872 INFO:teuthology.orchestra.run.smithi031.stdout:4564 write 0x50d60a9 thru 0x50d96eb (0x3643 bytes) 2024-09-16T22:17:22.876 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.886 INFO:teuthology.orchestra.run.smithi031.stdout:4566 write 0xd8546f9 thru 0xd861be1 (0xd4e9 bytes) 2024-09-16T22:17:22.892 INFO:teuthology.orchestra.run.smithi031.stdout:4567 read 0x725c016 thru 0x725c5d4 (0x5bf bytes) 2024-09-16T22:17:22.892 INFO:teuthology.orchestra.run.smithi031.stdout:4568 read 0x7bce2b1 thru 0x7bd4311 (0x6061 bytes) 2024-09-16T22:17:22.892 INFO:teuthology.orchestra.run.smithi031.stdout:4569 write 0x9cd2883 thru 0x9cdd880 (0xaffe bytes) 2024-09-16T22:17:22.895 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:22.897+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:22.897 INFO:teuthology.orchestra.run.smithi031.stdout:4572 trunc from 0xd861be2 to 0xc09b43a 2024-09-16T22:17:22.909 INFO:teuthology.orchestra.run.smithi031.stdout:4573 write 0xb9de203 thru 0xb9e2ab0 (0x48ae bytes) 2024-09-16T22:17:22.910 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:22.910 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:22.911 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:22.911 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:22.911 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:4575 punch from 0x34dccbd to 0x34e0b4a, (0x3e8d bytes) 2024-09-16T22:17:22.919 INFO:teuthology.orchestra.run.smithi031.stdout:4576 trunc from 0xc09b43a to 0xeba3ce5 2024-09-16T22:17:22.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:22.926 INFO:teuthology.orchestra.run.smithi031.stdout:4577 write 0x8ec5cc thru 0x8fb79d (0xf1d2 bytes) 2024-09-16T22:17:22.930 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.931 INFO:teuthology.orchestra.run.smithi031.stdout:4581 write 0x96ba923 thru 0x96bfcbb (0x5399 bytes) 2024-09-16T22:17:22.933 INFO:teuthology.orchestra.run.smithi031.stdout:4582 write 0xa4a450 thru 0xa52ce2 (0x8893 bytes) 2024-09-16T22:17:22.938 INFO:teuthology.orchestra.run.smithi031.stdout:4586 read 0xd2b1036 thru 0xd2b48f4 (0x38bf bytes) 2024-09-16T22:17:22.938 INFO:teuthology.orchestra.run.smithi031.stdout:4588 punch from 0x9f100ec to 0x9f1e44b, (0xe35f bytes) 2024-09-16T22:17:22.938 INFO:teuthology.orchestra.run.smithi031.stdout:4589 write 0x930c2ad thru 0x9318859 (0xc5ad bytes) 2024-09-16T22:17:22.941 INFO:teuthology.orchestra.run.smithi031.stdout:4590 write 0xc712b5e thru 0xc7146e7 (0x1b8a bytes) 2024-09-16T22:17:22.945 INFO:teuthology.orchestra.run.smithi031.stdout:4591 read 0xb2b3c33 thru 0xb2bd312 (0x96e0 bytes) 2024-09-16T22:17:22.945 INFO:teuthology.orchestra.run.smithi031.stdout:4592 read 0x599e999 thru 0x599ef09 (0x571 bytes) 2024-09-16T22:17:22.945 INFO:teuthology.orchestra.run.smithi031.stdout:4593 read 0x9fe27f5 thru 0x9fea318 (0x7b24 bytes) 2024-09-16T22:17:22.945 INFO:teuthology.orchestra.run.smithi031.stdout:4594 write 0xbfe795e thru 0xbff54f3 (0xdb96 bytes) 2024-09-16T22:17:22.952 INFO:teuthology.orchestra.run.smithi031.stdout:4597 write 0x643277b thru 0x6440d88 (0xe60e bytes) 2024-09-16T22:17:22.956 INFO:teuthology.orchestra.run.smithi031.stdout:4598 punch from 0xb8cbb0 to 0xb97d15, (0xb165 bytes) 2024-09-16T22:17:22.966 INFO:teuthology.orchestra.run.smithi031.stdout:4600 trunc from 0xeba3ce5 to 0x530a0af 2024-09-16T22:17:22.988 INFO:teuthology.orchestra.run.smithi031.stdout:4601 writesame 0x84de1c9 thru 0x84e2682 data_size 0x13(0x44ba bytes) 2024-09-16T22:17:22.989 INFO:teuthology.orchestra.run.smithi031.stdout:4602 read 0x10c3991 thru 0x10c71e7 (0x3857 bytes) 2024-09-16T22:17:22.989 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:22.996 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:22.997+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:22.996 INFO:teuthology.orchestra.run.smithi031.stdout:4604 write 0xa1fdf27 thru 0xa201fb4 (0x408e bytes) 2024-09-16T22:17:23.007 INFO:teuthology.orchestra.run.smithi031.stdout:4605 writesame 0xad03aa6 thru 0xad097c8 data_size 0x1(0x5d23 bytes) 2024-09-16T22:17:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:4608 trunc from 0xad097c9 to 0x655a08 2024-09-16T22:17:23.036 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:23.036 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:23.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:23.036 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.853582690000003, 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:17:23.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:17:23.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:23.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.803068001, 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:17:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.789741911, 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:17:23.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:17:23.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.734893386000003, 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:17:23.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:17:23.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42585 3.7 e51/23 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head v 51'156, mlcod=51'154)", 2024-09-16T22:17:23.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.865877365000003, 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036096402999999999, 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171395+0000", 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174324+0000", 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029293370000000002 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174349+0000", 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174358+0000", 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6979999999999993e-06 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174442+0000", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3858999999999998e-05 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174470+0000", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8042999999999999e-05 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207434+0000", 2024-09-16T22:17:23.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032963615000000002 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207490+0000", 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831999999999997e-05 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134175 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 811008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.267128316, 2024-09-16T22:17:23.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034372377000000003, 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134175 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:23.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770143+0000", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770145+0000", 2024-09-16T22:17:23.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1569999999999998e-06 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770151+0000", 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7640000000000002e-06 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770155+0000", 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5979999999999999e-06 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804368+0000", 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034212429000000003 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804423+0000", 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5794999999999997e-05 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804515+0000", 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1184e-05 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134177 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 833536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.267109606, 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034590927, 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134177 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:23.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770163+0000", 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.84e-06 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770169+0000", 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4439999999999999e-06 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770173+0000", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5140000000000003e-06 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804551+0000", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034377657999999998 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804601+0000", 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0064e-05 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804752+0000", 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015103999999999999 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134179 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 837632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670923509999996, 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034689483, 2024-09-16T22:17:23.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134179 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770179+0000", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.03e-07 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770180+0000", 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.612e-06 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770186+0000", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.378e-06 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770190+0000", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804780+0000", 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034590177 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804818+0000", 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8237e-05 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804868+0000", 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9496000000000002e-05 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134180 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 860160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670756289999998, 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034827098000000001, 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134180 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2899999999999999e-07 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770197+0000", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5349999999999999e-06 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770202+0000", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2970000000000003e-06 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770206+0000", 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3409999999999996e-06 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804904+0000", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034697027999999998 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804975+0000", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1922999999999996e-05 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805022+0000", 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6644999999999998e-05 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134182 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 882688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670589000000003, 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034913168000000001, 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134182 2024-09-16T22:17:23.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.34e-07 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770214+0000", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770219+0000", 2024-09-16T22:17:23.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2769999999999996e-06 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770223+0000", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4630000000000003e-06 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805048+0000", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034824695000000003 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805087+0000", 2024-09-16T22:17:23.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8609000000000003e-05 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805125+0000", 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8120000000000001e-05 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134183 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:23.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670418269999999, 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034987149000000002, 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134183 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-07 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770231+0000", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770236+0000", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770241+0000", 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4460000000000003e-06 2024-09-16T22:17:23.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805146+0000", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034905191000000002 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805181+0000", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5397e-05 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805216+0000", 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4644999999999998e-05 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134185 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670247679999997, 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035059762000000001, 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134185 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2899999999999999e-07 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770248+0000", 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770254+0000", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770259+0000", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8679999999999998e-06 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805236+0000", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034977038000000002 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805270+0000", 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3818e-05 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805306+0000", 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5556999999999999e-05 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134187 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 931840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:23.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2670060870000004, 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035128640000000003, 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134187 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:23.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770267+0000", 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.731e-06 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770272+0000", 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770276+0000", 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4660000000000001e-06 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805324+0000", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035047962000000002 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805357+0000", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2722000000000003e-05 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805393+0000", 2024-09-16T22:17:23.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6022000000000002e-05 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134188 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 935936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2669890080000004, 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035198934000000001, 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134188 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770284+0000", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5799999999999999e-06 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770289+0000", 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770293+0000", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3989999999999997e-06 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805414+0000", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035121195000000001 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805447+0000", 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2366999999999997e-05 2024-09-16T22:17:23.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805481+0000", 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3708999999999999e-05 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134189 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2669722280000002, 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035268075000000003, 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134189 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770299+0000", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999995e-07 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770301+0000", 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.668e-06 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770306+0000", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4029999999999998e-06 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770310+0000", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4009999999999999e-06 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805500+0000", 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035189486999999998 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805532+0000", 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2008999999999998e-05 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805566+0000", 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4582000000000002e-05 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134191 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2669551329999997, 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035337687, 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134191 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770316+0000", 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770317+0000", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6589999999999999e-06 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770323+0000", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770328+0000", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1719999999999998e-06 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805586+0000", 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035257976000000003 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805618+0000", 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2227000000000002e-05 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805653+0000", 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4771000000000003e-05 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134192 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 985088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2669369360000005, 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035415580000000002, 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134192 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.114 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770340+0000", 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.592e-06 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770345+0000", 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770350+0000", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805674+0000", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035324384 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805711+0000", 2024-09-16T22:17:23.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6541999999999997e-05 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805749+0000", 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8509e-05 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134194 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.266914152, 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035494879999999999, 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134194 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770357+0000", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770359+0000", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0650000000000001e-06 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770365+0000", 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9349999999999999e-06 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770370+0000", 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8300000000000003e-06 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805771+0000", 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035400993999999998 2024-09-16T22:17:23.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805806+0000", 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5466000000000002e-05 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805851+0000", 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5179000000000002e-05 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134195 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2668949769999998, 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035569861000000001, 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134195 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8299999999999998e-07 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770378+0000", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8789999999999999e-06 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770384+0000", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7030000000000003e-06 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770388+0000", 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7659999999999998e-06 2024-09-16T22:17:23.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805873+0000", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035484276000000002 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805909+0000", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6155999999999997e-05 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805946+0000", 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6698e-05 2024-09-16T22:17:23.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134197 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1034240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2668766309999997, 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035639397000000003, 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.127 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134197 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5400000000000002e-07 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770396+0000", 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6619999999999999e-06 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770401+0000", 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770406+0000", 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805959+0000", 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035553478999999999 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805995+0000", 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6214000000000003e-05 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.806033+0000", 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7914000000000003e-05 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.131 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.132 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:23.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:23.134 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:23.134+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:23.135 INFO:teuthology.orchestra.run.smithi031.stdout:4610 trunc from 0x655a08 to 0x9ef7283 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4611 punch from 0x25bbd6d to 0x25c6ede, (0xb171 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4612 read 0x7be427f thru 0x7bf281c (0xe59e bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4613 trunc from 0x9ef7283 to 0x1633436 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4614 write 0x26276b1 thru 0x2632f02 (0xb852 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4615 read 0xfebd10 thru 0xffabf7 (0xeee8 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4616 punch from 0x34ee4b to 0x351c70, (0x2e25 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4617 punch from 0x20989a5 to 0x20a6cb6, (0xe311 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4618 write 0x35138c thru 0x357348 (0x5fbd bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4619 read 0x657a28 thru 0x665594 (0xdb6d bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4620 write 0x231ca3b thru 0x2327fec (0xb5b2 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4621 read 0x5f1532 thru 0x5fdc06 (0xc6d5 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4623 writesame 0xad4211a thru 0xad4dc6a data_size 0x1(0xbb51 bytes) 2024-09-16T22:17:23.136 INFO:teuthology.orchestra.run.smithi031.stdout:4624 writesame 0x42a9742 thru 0x42b7297 data_size 0x19(0xdb56 bytes) 2024-09-16T22:17:23.137 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.137 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.137 INFO:teuthology.orchestra.run.smithi031.stdout:4629 write 0x83d8a49 thru 0x83daaed (0x20a5 bytes) 2024-09-16T22:17:23.137 INFO:teuthology.orchestra.run.smithi031.stdout:4630 punch from 0xa13cc49 to 0xa140aa1, (0x3e58 bytes) 2024-09-16T22:17:23.137 INFO:teuthology.orchestra.run.smithi031.stdout:4631 write 0xc27baee thru 0xc28201e (0x6531 bytes) 2024-09-16T22:17:23.142 INFO:teuthology.orchestra.run.smithi031.stdout:4632 trunc from 0xc28201f to 0x907d4f6 2024-09-16T22:17:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:4633 punch from 0x69a88e6 to 0x69b2740, (0x9e5a bytes) 2024-09-16T22:17:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:4634 read 0x3a46a5 thru 0x3b364d (0xefa9 bytes) 2024-09-16T22:17:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:4637 read 0x7003e1d thru 0x7009542 (0x5726 bytes) 2024-09-16T22:17:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.153 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.160 INFO:teuthology.orchestra.run.smithi031.stdout:4641 write 0xa834352 thru 0xa841e14 (0xdac3 bytes) 2024-09-16T22:17:23.164 INFO:teuthology.orchestra.run.smithi031.stdout:4642 writesame 0x18d8416 thru 0x18e579c data_size 0x1(0xd387 bytes) 2024-09-16T22:17:23.178 INFO:teuthology.orchestra.run.smithi031.stdout:4643 write 0x550439 thru 0x550ab3 (0x67b bytes) 2024-09-16T22:17:23.180 INFO:teuthology.orchestra.run.smithi031.stdout:4644 writesame 0x1214f82 thru 0x12176e2 data_size 0x1(0x2761 bytes) 2024-09-16T22:17:23.193 INFO:teuthology.orchestra.run.smithi031.stdout:4647 trunc from 0xa841e15 to 0x57914a1 2024-09-16T22:17:23.204 INFO:teuthology.orchestra.run.smithi031.stdout:4648 read 0x1f22c08 thru 0x1f31e39 (0xf232 bytes) 2024-09-16T22:17:23.204 INFO:teuthology.orchestra.run.smithi031.stdout:4651 writesame 0x220b15f thru 0x2211548 data_size 0x62(0x63ea bytes) 2024-09-16T22:17:23.208 INFO:teuthology.orchestra.run.smithi031.stdout:4652 punch from 0xacba2f to 0xad75b1, (0xbb82 bytes) 2024-09-16T22:17:23.218 INFO:teuthology.orchestra.run.smithi031.stdout:4654 write 0x8c79102 thru 0x8c8717a (0xe079 bytes) 2024-09-16T22:17:23.222 INFO:teuthology.orchestra.run.smithi031.stdout:4655 punch from 0x7a1f37b to 0x7a20b43, (0x17c8 bytes) 2024-09-16T22:17:23.222 INFO:teuthology.orchestra.run.smithi031.stdout:4657 read 0x1262c6f thru 0x1269b28 (0x6eba bytes) 2024-09-16T22:17:23.223 INFO:teuthology.orchestra.run.smithi031.stdout:4658 read 0x16e9ad2 thru 0x16f89d3 (0xef02 bytes) 2024-09-16T22:17:23.223 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.223 INFO:teuthology.orchestra.run.smithi031.stdout:4661 punch from 0x6cdf062 to 0x6ce4cb8, (0x5c56 bytes) 2024-09-16T22:17:23.223 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.223 INFO:teuthology.orchestra.run.smithi031.stdout:4664 trunc from 0x8c8717b to 0x9a24031 2024-09-16T22:17:23.233 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:23.235+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:23.239 INFO:teuthology.orchestra.run.smithi031.stdout:4665 trunc from 0x9a24031 to 0x92b03a 2024-09-16T22:17:23.252 INFO:teuthology.orchestra.run.smithi031.stdout:4666 punch from 0x101fff to 0x103c16, (0x1c17 bytes) 2024-09-16T22:17:23.252 INFO:teuthology.orchestra.run.smithi031.stdout:4667 read 0x5f4e2c thru 0x602acd (0xdca2 bytes) 2024-09-16T22:17:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:17:23.272 INFO:teuthology.orchestra.run.smithi031.stdout:4668 writesame 0xbd22833 thru 0xbd257ca data_size 0x4(0x2f98 bytes) 2024-09-16T22:17:23.276 INFO:teuthology.orchestra.run.smithi031.stdout:4669 trunc from 0xbd257cb to 0x883ec6b 2024-09-16T22:17:23.287 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:23.287 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:23.289 INFO:teuthology.orchestra.run.smithi031.stdout:4670 write 0x8a59710 thru 0x8a5f7f1 (0x60e2 bytes) 2024-09-16T22:17:23.294 INFO:teuthology.orchestra.run.smithi031.stdout:4671 read 0x4dc8099 thru 0x4dc8301 (0x269 bytes) 2024-09-16T22:17:23.295 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.295 INFO:teuthology.orchestra.run.smithi031.stdout:4673 write 0x1a62b65 thru 0x1a62eef (0x38b bytes) 2024-09-16T22:17:23.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:23.297 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.297 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.297 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.305 INFO:teuthology.orchestra.run.smithi031.stdout:4679 write 0x9f2789b thru 0x9f29d6b (0x24d1 bytes) 2024-09-16T22:17:23.310 INFO:teuthology.orchestra.run.smithi031.stdout:4680 read 0x1717920 thru 0x172283f (0xaf20 bytes) 2024-09-16T22:17:23.313 INFO:teuthology.orchestra.run.smithi031.stdout:4681 trunc from 0x9f29d6c to 0xcb227a8 2024-09-16T22:17:23.326 INFO:teuthology.orchestra.run.smithi031.stdout:4682 write 0xd1f5cd7 thru 0xd200027 (0xa351 bytes) 2024-09-16T22:17:23.333 INFO:teuthology.orchestra.run.smithi031.stdout:4684 write 0x4173d3d thru 0x4176fba (0x327e bytes) 2024-09-16T22:17:23.334 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:23.335+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:23.337 INFO:teuthology.orchestra.run.smithi031.stdout:4685 write 0x15f004d thru 0x15ffb70 (0xfb24 bytes) 2024-09-16T22:17:23.340 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.340 INFO:teuthology.orchestra.run.smithi031.stdout:4689 read 0x6950377 thru 0x69507ec (0x476 bytes) 2024-09-16T22:17:23.340 INFO:teuthology.orchestra.run.smithi031.stdout:4691 writesame 0x53d957d thru 0x53daa38 data_size 0x2(0x14bc bytes) 2024-09-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi031.stdout:4692 writesame 0x2323543 thru 0x2325767 data_size 0x1(0x2225 bytes) 2024-09-16T22:17:23.350 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:23.350 INFO:teuthology.orchestra.run.smithi031.stdout:4694 punch from 0x3dcab34 to 0x3dd01df, (0x56ab bytes) 2024-09-16T22:17:23.350 INFO:teuthology.orchestra.run.smithi031.stdout:4695 write 0xd5a4c4 thru 0xd679bb (0xd4f8 bytes) 2024-09-16T22:17:23.363 INFO:teuthology.orchestra.run.smithi031.stdout:4697 trunc from 0xd200028 to 0x5eeed22 2024-09-16T22:17:23.373 INFO:teuthology.orchestra.run.smithi031.stdout:4698 write 0x50b147c thru 0x50bcc11 (0xb796 bytes) 2024-09-16T22:17:23.381 INFO:teuthology.orchestra.run.smithi031.stdout:4699 trunc from 0x5eeed22 to 0xa7843ca 2024-09-16T22:17:23.386 INFO:teuthology.orchestra.run.smithi031.stdout:4700 read 0x48290e1 thru 0x4834c14 (0xbb34 bytes) 2024-09-16T22:17:23.394 INFO:teuthology.orchestra.run.smithi031.stdout:4701 write 0xc31c0bc thru 0xc31c8aa (0x7ef bytes) 2024-09-16T22:17:23.406 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:23.407 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:23.407 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:23.407 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:23.407 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:23.409 INFO:teuthology.orchestra.run.smithi031.stdout:4702 trunc from 0xc31c8ab to 0x1ec9487 2024-09-16T22:17:23.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:23.442 INFO:teuthology.orchestra.run.smithi031.stdout:4703 writesame 0xda490d7 thru 0xda52c43 data_size 0x3(0x9b6d bytes) 2024-09-16T22:17:23.449 INFO:teuthology.orchestra.run.smithi031.stdout:4706 write 0xdf9da81 thru 0xdfa18f6 (0x3e76 bytes) 2024-09-16T22:17:23.453 INFO:teuthology.orchestra.run.smithi031.stdout:4707 writesame 0xcc92bcb thru 0xcca1758 data_size 0xb(0xeb8e bytes) 2024-09-16T22:17:23.458 INFO:teuthology.orchestra.run.smithi031.stdout:4708 write 0x5fd7e52 thru 0x5fdb5f3 (0x37a2 bytes) 2024-09-16T22:17:23.459 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:23.460+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:23.461 INFO:teuthology.orchestra.run.smithi031.stdout:4709 clone 38 order 23 su 4194304 sc 2 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.748015972999999, 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:17:23.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747995813999999, 2024-09-16T22:17:23.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:23.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:17:23.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747976773, 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.537 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747941365999999, 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:17:23.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747922204, 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:17:23.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747902922, 2024-09-16T22:17:23.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:17:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747884224, 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747866488, 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:17:23.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:17:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747848857999999, 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747829639000001, 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:23.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:17:23.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.568 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747812539, 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:23.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:17:23.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747793803, 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:23.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747769955999999, 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:23.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:17:23.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747750977999999, 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.582 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747731862, 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:17:23.586 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:23.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:17:23.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747701609, 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.747680808, 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.595 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:23.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.722104054999999, 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.722086477, 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.721068913, 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:17:23.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:23.614 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:23.615+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:23.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:23.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:23.714 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:23.715+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:23.723 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:23.723 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:23.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:23.816 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:23.818+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:23.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:23.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:23.868 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:23.881 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:23.881 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:23.884 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:23.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:23.901 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:23.901 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:23.918 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:23.919 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:23.920+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:23.935 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:23.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:24.009 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.010 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.017 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:24.018+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:24.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:24.118 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:24.120+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:24.166 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.166 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:24.219 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:24.221+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:24.320 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:24.320+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:24.406 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.407 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:24.459 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:24.461+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:24.522 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:24.523+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:24.529 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.529 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.532 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T22:17:24.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:24.623 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:24.623+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:24.724 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:24.725+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:24.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:24.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:24.824 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:24.825+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:24.926 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:24.927+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:25.041 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:25.043+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:25.127 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:25.129+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:25.184 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.184 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:25.228 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:25.229+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:25.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:17:25.307 INFO:teuthology.orchestra.run.smithi031.stdout:4710 write 0xccab50b thru 0xccb7be8 (0xc6de bytes) 2024-09-16T22:17:25.307 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.307 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:25.319 INFO:teuthology.orchestra.run.smithi031.stdout:4711 writesame 0xcb2c551 thru 0xcb3a6b0 data_size 0x30(0xe160 bytes) 2024-09-16T22:17:25.328 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:25.329+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:25.336 INFO:teuthology.orchestra.run.smithi031.stdout:4712 trunc from 0xdfa18f7 to 0x1e47a4c 2024-09-16T22:17:25.364 INFO:teuthology.orchestra.run.smithi031.stdout:4713 writesame 0x71d68c5 thru 0x71e0788 data_size 0x12(0x9ec4 bytes) 2024-09-16T22:17:25.379 INFO:teuthology.orchestra.run.smithi031.stdout:4714 writesame 0xea2cb4b thru 0xea35606 data_size 0x4(0x8abc bytes) 2024-09-16T22:17:25.379 INFO:teuthology.orchestra.run.smithi031.stdout:4715 read 0x9784565 thru 0x9790178 (0xbc14 bytes) 2024-09-16T22:17:25.379 INFO:teuthology.orchestra.run.smithi031.stdout:4717 write 0x7f850db thru 0x7f89b93 (0x4ab9 bytes) 2024-09-16T22:17:25.395 INFO:teuthology.orchestra.run.smithi031.stdout:4719 trunc from 0xea35607 to 0x3c8cde2 2024-09-16T22:17:25.406 INFO:teuthology.orchestra.run.smithi031.stdout:4720 punch from 0x225a622 to 0x22623fb, (0x7dd9 bytes) 2024-09-16T22:17:25.415 INFO:teuthology.orchestra.run.smithi031.stdout:4721 write 0x625bd85 thru 0x626963e (0xd8ba bytes) 2024-09-16T22:17:25.418 INFO:teuthology.orchestra.run.smithi031.stdout:4723 punch from 0x42572aa to 0x425bc0d, (0x4963 bytes) 2024-09-16T22:17:25.430 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:25.432+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:25.435 INFO:teuthology.orchestra.run.smithi031.stdout:4724 write 0xecb536a thru 0xecc1503 (0xc19a bytes) 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:4727 read 0xd06ae1 thru 0xd15d50 (0xf270 bytes) 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:4728 read 0x9331664 thru 0x933bc6c (0xa609 bytes) 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:4729 read 0x4f33b5a thru 0x4f40c05 (0xd0ac bytes) 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:4730 read 0xe54b4fb thru 0xe556a95 (0xb59b bytes) 2024-09-16T22:17:25.439 INFO:teuthology.orchestra.run.smithi031.stdout:4731 read 0x2779f11 thru 0x2782aa8 (0x8b98 bytes) 2024-09-16T22:17:25.451 INFO:teuthology.orchestra.run.smithi031.stdout:4732 trunc from 0xecc1504 to 0x3678f30 2024-09-16T22:17:25.468 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.468 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.480 INFO:teuthology.orchestra.run.smithi031.stdout:4734 writesame 0xe837ca8 thru 0xe843031 data_size 0x62(0xb38a bytes) 2024-09-16T22:17:25.480 INFO:teuthology.orchestra.run.smithi031.stdout:4736 punch from 0x4025abb to 0x402cde7, (0x732c bytes) 2024-09-16T22:17:25.480 INFO:teuthology.orchestra.run.smithi031.stdout:4738 writesame 0xd383164 thru 0xd386ee8 data_size 0x1(0x3d85 bytes) 2024-09-16T22:17:25.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:25.489 INFO:teuthology.orchestra.run.smithi031.stdout:4739 writesame 0xd22e984 thru 0xd23dc21 data_size 0x5(0xf29e bytes) 2024-09-16T22:17:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:4741 read 0xcbaf8a1 thru 0xcbb2094 (0x27f4 bytes) 2024-09-16T22:17:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:4742 read 0x3758d95 thru 0x3758e41 (0xad bytes) 2024-09-16T22:17:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.496 INFO:teuthology.orchestra.run.smithi031.stdout:4745 write 0x666e620 thru 0x667711c (0x8afd bytes) 2024-09-16T22:17:25.513 INFO:teuthology.orchestra.run.smithi031.stdout:4746 trunc from 0xe843032 to 0xf7f530 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:4748 punch from 0xf0a92 to 0xfaecc, (0xa43a bytes) 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:4749 punch from 0x60b9cc to 0x613327, (0x795b bytes) 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:4751 read 0x53b871 thru 0x54aab9 (0xf249 bytes) 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:4753 read 0x6e35f8 thru 0x6f0517 (0xcf20 bytes) 2024-09-16T22:17:25.529 INFO:teuthology.orchestra.run.smithi031.stdout:4754 punch from 0x469742 to 0x4787aa, (0xf068 bytes) 2024-09-16T22:17:25.530 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:25.532+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:25.540 INFO:teuthology.orchestra.run.smithi031.stdout:4755 writesame 0x65ac791 thru 0x65b8934 data_size 0xa2(0xc1a4 bytes) 2024-09-16T22:17:25.540 INFO:teuthology.orchestra.run.smithi031.stdout:4757 punch from 0x5518645 to 0x55239ca, (0xb385 bytes) 2024-09-16T22:17:25.540 INFO:teuthology.orchestra.run.smithi031.stdout:4758 write 0x13c80e thru 0x141886 (0x5079 bytes) 2024-09-16T22:17:25.568 INFO:teuthology.orchestra.run.smithi031.stdout:4759 writesame 0xd186d64 thru 0xd18cdc9 data_size 0x1b(0x6066 bytes) 2024-09-16T22:17:25.568 INFO:teuthology.orchestra.run.smithi031.stdout:4760 read 0x87000fd thru 0x870234e (0x2252 bytes) 2024-09-16T22:17:25.569 INFO:teuthology.orchestra.run.smithi031.stdout:4761 read 0x25f978a thru 0x25fc1da (0x2a51 bytes) 2024-09-16T22:17:25.569 INFO:teuthology.orchestra.run.smithi031.stdout:4762 write 0xbd4d9b1 thru 0xbd55f77 (0x85c7 bytes) 2024-09-16T22:17:25.571 INFO:teuthology.orchestra.run.smithi031.stdout:4763 punch from 0x66f0977 to 0x66f3020, (0x26a9 bytes) 2024-09-16T22:17:25.572 INFO:teuthology.orchestra.run.smithi031.stdout:4765 trunc from 0xd18cdca to 0xda673ea 2024-09-16T22:17:25.577 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.578 INFO:teuthology.orchestra.run.smithi031.stdout:4767 read 0x8e4bc23 thru 0x8e516d1 (0x5aaf bytes) 2024-09-16T22:17:25.578 INFO:teuthology.orchestra.run.smithi031.stdout:4768 read 0x713243e thru 0x713c937 (0xa4fa bytes) 2024-09-16T22:17:25.579 INFO:teuthology.orchestra.run.smithi031.stdout:4770 trunc from 0xda673ea to 0xc7e6b1f 2024-09-16T22:17:25.584 INFO:teuthology.orchestra.run.smithi031.stdout:4772 punch from 0x8751300 to 0x875271e, (0x141e bytes) 2024-09-16T22:17:25.587 INFO:teuthology.orchestra.run.smithi031.stdout:4775 trunc from 0xc7e6b1f to 0xe6e8af7 2024-09-16T22:17:25.587 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.587 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.594 INFO:teuthology.orchestra.run.smithi031.stdout:4777 write 0x8f4cd0c thru 0x8f517d3 (0x4ac8 bytes) 2024-09-16T22:17:25.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:25.597 INFO:teuthology.orchestra.run.smithi031.stdout:4778 write 0x766185c thru 0x766aa18 (0x91bd bytes) 2024-09-16T22:17:25.600 INFO:teuthology.orchestra.run.smithi031.stdout:4779 writesame 0x1a09397 thru 0x1a0e6b3 data_size 0x1(0x531d bytes) 2024-09-16T22:17:25.613 INFO:teuthology.orchestra.run.smithi031.stdout:4780 write 0x13f01bd thru 0x13fc43e (0xc282 bytes) 2024-09-16T22:17:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:4781 punch from 0x8f1bfcb to 0x8f1d208, (0x123d bytes) 2024-09-16T22:17:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:4782 write 0x952d669 thru 0x952fd14 (0x26ac bytes) 2024-09-16T22:17:25.618 INFO:teuthology.orchestra.run.smithi031.stdout:4783 write 0x98c9c9c thru 0x98d3f44 (0xa2a9 bytes) 2024-09-16T22:17:25.622 INFO:teuthology.orchestra.run.smithi031.stdout:4784 punch from 0x6318475 to 0x6321eed, (0x9a78 bytes) 2024-09-16T22:17:25.622 INFO:teuthology.orchestra.run.smithi031.stdout:4785 punch from 0x4f727d1 to 0x4f82676, (0xfea5 bytes) 2024-09-16T22:17:25.622 INFO:teuthology.orchestra.run.smithi031.stdout:4787 write 0x8e7660a thru 0x8e843db (0xddd2 bytes) 2024-09-16T22:17:25.623 INFO:teuthology.orchestra.run.smithi031.stdout:4788 write 0x1e41b72 thru 0x1e4fe04 (0xe293 bytes) 2024-09-16T22:17:25.632 INFO:teuthology.orchestra.run.smithi031.stdout:4789 trunc from 0xe6e8af7 to 0x8f6565b 2024-09-16T22:17:25.637 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:25.639+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:25.643 INFO:teuthology.orchestra.run.smithi031.stdout:4790 write 0x5383297 thru 0x5392ce7 (0xfa51 bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4791 read 0x3b94990 thru 0x3b97ebd (0x352e bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4793 read 0x1d85650 thru 0x1d86e10 (0x17c1 bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4794 punch from 0x64a3b33 to 0x64a4342, (0x80f bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4795 punch from 0x27e88ea to 0x27ed70d, (0x4e23 bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4796 read 0x7b6b600 thru 0x7b6e51b (0x2f1c bytes) 2024-09-16T22:17:25.648 INFO:teuthology.orchestra.run.smithi031.stdout:4798 write 0x2fe29e3 thru 0x2fe9efb (0x7519 bytes) 2024-09-16T22:17:25.658 INFO:teuthology.orchestra.run.smithi031.stdout:4800 writesame 0xa646153 thru 0xa651021 data_size 0x7(0xaecf bytes) 2024-09-16T22:17:25.658 INFO:teuthology.orchestra.run.smithi031.stdout:4801 write 0x9741705 thru 0x974a2bb (0x8bb7 bytes) 2024-09-16T22:17:25.666 INFO:teuthology.orchestra.run.smithi031.stdout:4802 trunc from 0xa651022 to 0x5502abc 2024-09-16T22:17:25.677 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.677 INFO:teuthology.orchestra.run.smithi031.stdout:4804 read 0x5bbeb2 thru 0x5c7071 (0xb1c0 bytes) 2024-09-16T22:17:25.682 INFO:teuthology.orchestra.run.smithi031.stdout:4805 trunc from 0x5502abc to 0x95c85be 2024-09-16T22:17:25.686 INFO:teuthology.orchestra.run.smithi031.stdout:4807 punch from 0x6758789 to 0x6759eb7, (0x172e bytes) 2024-09-16T22:17:25.696 INFO:teuthology.orchestra.run.smithi031.stdout:4808 trunc from 0x95c85be to 0xb51b03 2024-09-16T22:17:25.725 INFO:teuthology.orchestra.run.smithi031.stdout:4809 write 0xbfbf9f0 thru 0xbfcab57 (0xb168 bytes) 2024-09-16T22:17:25.728 INFO:teuthology.orchestra.run.smithi031.stdout:4811 punch from 0x6969d8f to 0x69784fb, (0xe76c bytes) 2024-09-16T22:17:25.729 INFO:teuthology.orchestra.run.smithi031.stdout:4812 read 0x75a9727 thru 0x75ae0ce (0x49a8 bytes) 2024-09-16T22:17:25.733 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:25.734+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:25.735 INFO:teuthology.orchestra.run.smithi031.stdout:4813 writesame 0xd43b52f thru 0xd446e93 data_size 0x1(0xb965 bytes) 2024-09-16T22:17:25.739 INFO:teuthology.orchestra.run.smithi031.stdout:4814 trunc from 0xd446e94 to 0x97444b9 2024-09-16T22:17:25.740 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.740 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:25.749 INFO:teuthology.orchestra.run.smithi031.stdout:4815 read 0x352aa1e thru 0x3534409 (0x99ec bytes) 2024-09-16T22:17:25.749 INFO:teuthology.orchestra.run.smithi031.stdout:4817 writesame 0x53f1c32 thru 0x53ff28f data_size 0x17(0xd65e bytes) 2024-09-16T22:17:25.753 INFO:teuthology.orchestra.run.smithi031.stdout:4819 punch from 0x7dc257d to 0x7dd03ca, (0xde4d bytes) 2024-09-16T22:17:25.753 INFO:teuthology.orchestra.run.smithi031.stdout:4820 punch from 0x50cbad0 to 0x50cf64e, (0x3b7e bytes) 2024-09-16T22:17:25.753 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:25.753 INFO:teuthology.orchestra.run.smithi031.stdout:4823 punch from 0x1f64c58 to 0x1f67ece, (0x3276 bytes) 2024-09-16T22:17:25.753 INFO:teuthology.orchestra.run.smithi031.stdout:4824 punch from 0xd2d5d to 0xd7a91, (0x4d34 bytes) 2024-09-16T22:17:25.754 INFO:teuthology.orchestra.run.smithi031.stdout:4825 punch from 0x94fadcc to 0x9509a2a, (0xec5e bytes) 2024-09-16T22:17:25.754 INFO:teuthology.orchestra.run.smithi031.stdout:4826 write 0x1cf75ef thru 0x1cfbaef (0x4501 bytes) 2024-09-16T22:17:25.763 INFO:teuthology.orchestra.run.smithi031.stdout:4827 trunc from 0x97444b9 to 0x264fcb9 2024-09-16T22:17:25.788 INFO:teuthology.orchestra.run.smithi031.stdout:4828 write 0xbd112b3 thru 0xbd1cf06 (0xbc54 bytes) 2024-09-16T22:17:25.798 INFO:teuthology.orchestra.run.smithi031.stdout:4829 trunc from 0xbd1cf07 to 0x53a6c55 2024-09-16T22:17:25.807 INFO:teuthology.orchestra.run.smithi031.stdout:4830 punch from 0x36488ec to 0x3652c42, (0xa356 bytes) 2024-09-16T22:17:25.807 INFO:teuthology.orchestra.run.smithi031.stdout:4831 write 0x529cc7f thru 0x529ed38 (0x20ba bytes) 2024-09-16T22:17:25.817 INFO:teuthology.orchestra.run.smithi031.stdout:4832 trunc from 0x53a6c55 to 0xbf29bbb 2024-09-16T22:17:25.822 INFO:teuthology.orchestra.run.smithi031.stdout:4833 write 0x927eaf3 thru 0x9286395 (0x78a3 bytes) 2024-09-16T22:17:25.831 INFO:teuthology.orchestra.run.smithi031.stdout:4834 trunc from 0xbf29bbb to 0x67e6998 2024-09-16T22:17:25.834 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:25.835+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:25.841 INFO:teuthology.orchestra.run.smithi031.stdout:4837 trunc from 0x67e6998 to 0x75fb0f8 2024-09-16T22:17:25.853 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.853 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.857 INFO:teuthology.orchestra.run.smithi031.stdout:4839 write 0xc1e9772 thru 0xc1ef84f (0x60de bytes) 2024-09-16T22:17:25.860 INFO:teuthology.orchestra.run.smithi031.stdout:4840 clone 39 order 22 su 16384 sc 9 2024-09-16T22:17:25.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:25.935 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:25.937+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:25.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:25.964 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:25.965 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:25.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:26.057 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:26.059+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:26.089 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:26.090 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:26.106 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:26.107 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:26.117 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:26.117 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:26.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:26.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:26.136 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:26.138+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:26.139 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:26.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:26.189 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:26.190 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:26.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:26.206 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:26.237 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:26.239+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.765943857, 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764398079999999, 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764379999999999, 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764363438, 2024-09-16T22:17:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.76434695, 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764329878000002, 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764313372, 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.764283580000001, 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759951322999999, 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759929062000001, 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759908068000001, 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759887320000001, 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759867018000001, 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759847014999998, 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759826889999999, 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759807295000002, 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759786373000001, 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759765585, 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759743923999999, 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.759714834, 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.360 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:17:26.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:26.363 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone38 from 0xc1ef850 (overlap 0xb51b03) to 0x5a8d81 2024-09-16T22:17:26.363 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:26.363+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:26.409 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T22:17:26.463 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:26.464+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:26.509 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:26.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:26.509 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:26.509 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:26.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:26.563 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:26.564+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:26.621 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:26.621 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:26.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:4844 punch from 0x806236f to 0x806ec43, (0xc8d4 bytes) 2024-09-16T22:17:26.664 INFO:teuthology.orchestra.run.smithi031.stdout:4845 punch from 0x9acd822 to 0x9ad0022, (0x2800 bytes) 2024-09-16T22:17:26.664 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:26.664 INFO:teuthology.orchestra.run.smithi031.stdout:4847 read 0xbaa11f4 thru 0xbaad6d2 (0xc4df bytes) 2024-09-16T22:17:26.664 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:26.664 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:26.664 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T22:17:26.664 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:26.664 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:26.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-16T22:17:26.669 INFO:teuthology.orchestra.run.smithi031.stdout:4850 trunc from 0xc1ef850 to 0x6dffcff 2024-09-16T22:17:26.674 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:26.676+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:26.692 INFO:teuthology.orchestra.run.smithi031.stdout:4851 read 0x1940049 thru 0x194bb7c (0xbb34 bytes) 2024-09-16T22:17:26.692 INFO:teuthology.orchestra.run.smithi031.stdout:4853 write 0x57e402f thru 0x57ef119 (0xb0eb bytes) 2024-09-16T22:17:26.726 INFO:teuthology.orchestra.run.smithi031.stdout:4854 trunc from 0x6dffcff to 0xa8e3af1 2024-09-16T22:17:26.732 INFO:teuthology.orchestra.run.smithi031.stdout:4855 write 0x1411e51 thru 0x141c996 (0xab46 bytes) 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:26.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.574828791999998, 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:26.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:17:26.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:17:26.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.524314103000002, 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:17:26.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:17:26.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.510988013000002, 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:17:26.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:17:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.456139487999998, 2024-09-16T22:17:26.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:26.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:17:26.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:26.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:42585 3.7 e51/23 3:e2161cfc:::rbd_data.11cd49ca855d.0000000000000000:head v 51'156, mlcod=51'154)", 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.587123466999998, 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036096402999999999, 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171393+0000", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.171395+0000", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2780000000000001e-06 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174324+0000", 2024-09-16T22:17:26.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029293370000000002 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174349+0000", 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4740999999999999e-05 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174358+0000", 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6979999999999993e-06 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174442+0000", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3858999999999998e-05 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.174470+0000", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8042999999999999e-05 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207434+0000", 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032963615000000002 2024-09-16T22:17:26.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:49.207490+0000", 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5831999999999997e-05 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134175 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 811008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9883744180000003, 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034372377000000003, 2024-09-16T22:17:26.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134175 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770142+0000", 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770143+0000", 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770145+0000", 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1569999999999998e-06 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770151+0000", 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7640000000000002e-06 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770155+0000", 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5979999999999999e-06 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804368+0000", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034212429000000003 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804423+0000", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5794999999999997e-05 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804515+0000", 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1184e-05 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134177 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 833536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9883557080000003, 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034590927, 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134177 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770161+0000", 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:26.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770163+0000", 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.84e-06 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770169+0000", 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4439999999999999e-06 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770173+0000", 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5140000000000003e-06 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804551+0000", 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034377657999999998 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804601+0000", 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0064e-05 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804752+0000", 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015103999999999999 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134179 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 837632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9883384529999999, 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034689483, 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134179 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770178+0000", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770179+0000", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.03e-07 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770180+0000", 2024-09-16T22:17:26.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.612e-06 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770186+0000", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.378e-06 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770190+0000", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804780+0000", 2024-09-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034590177 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804818+0000", 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8237e-05 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804868+0000", 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9496000000000002e-05 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134180 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 860160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9883217310000001, 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034827098000000001, 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134180 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770195+0000", 2024-09-16T22:17:26.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2899999999999999e-07 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770197+0000", 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5349999999999999e-06 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770202+0000", 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2970000000000003e-06 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770206+0000", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3409999999999996e-06 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804904+0000", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034697027999999998 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.804975+0000", 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1922999999999996e-05 2024-09-16T22:17:26.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805022+0000", 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6644999999999998e-05 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134182 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 882688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9883050019999997, 2024-09-16T22:17:26.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034913168000000001, 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134182 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:26.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770212+0000", 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.34e-07 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770214+0000", 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770219+0000", 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2769999999999996e-06 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770223+0000", 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4630000000000003e-06 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805048+0000", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034824695000000003 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805087+0000", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8609000000000003e-05 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805125+0000", 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8120000000000001e-05 2024-09-16T22:17:26.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134183 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 886784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9882879290000002, 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034987149000000002, 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134183 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770229+0000", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-07 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770231+0000", 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770236+0000", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5060000000000003e-06 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770241+0000", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4460000000000003e-06 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805146+0000", 2024-09-16T22:17:26.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034905191000000002 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805181+0000", 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5397e-05 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805216+0000", 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4644999999999998e-05 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134185 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 909312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.98827087, 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035059762000000001, 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134185 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770246+0000", 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2899999999999999e-07 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770248+0000", 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770254+0000", 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770259+0000", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8679999999999998e-06 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805236+0000", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034977038000000002 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805270+0000", 2024-09-16T22:17:26.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3818e-05 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805306+0000", 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5556999999999999e-05 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134187 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 931840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:26.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9882521889999998, 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035128640000000003, 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134187 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770265+0000", 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770267+0000", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.731e-06 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770272+0000", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4e-06 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770276+0000", 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4660000000000001e-06 2024-09-16T22:17:26.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805324+0000", 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035047962000000002 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805357+0000", 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2722000000000003e-05 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805393+0000", 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6022000000000002e-05 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134188 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 935936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9882351099999998, 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035198934000000001, 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134188 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770282+0000", 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770284+0000", 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5799999999999999e-06 2024-09-16T22:17:26.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770289+0000", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770293+0000", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3989999999999997e-06 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805414+0000", 2024-09-16T22:17:26.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035121195000000001 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805447+0000", 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2366999999999997e-05 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805481+0000", 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3708999999999999e-05 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134189 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9882183300000005, 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035268075000000003, 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134189 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770298+0000", 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770299+0000", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999995e-07 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770301+0000", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.668e-06 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770306+0000", 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4029999999999998e-06 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770310+0000", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4009999999999999e-06 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805500+0000", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035189486999999998 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805532+0000", 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2008999999999998e-05 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805566+0000", 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4582000000000002e-05 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134191 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.988201235, 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035337687, 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134191 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770315+0000", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770316+0000", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770317+0000", 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6589999999999999e-06 2024-09-16T22:17:26.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770323+0000", 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5489999999999997e-06 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770328+0000", 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1719999999999998e-06 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805586+0000", 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035257976000000003 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805618+0000", 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2227000000000002e-05 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805653+0000", 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4771000000000003e-05 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.840 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134192 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 985088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9881830379999998, 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035415580000000002, 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134192 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770334+0000", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770340+0000", 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.592e-06 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770345+0000", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770350+0000", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805674+0000", 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035324384 2024-09-16T22:17:26.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805711+0000", 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6541999999999997e-05 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805749+0000", 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8509e-05 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134194 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9881602540000003, 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035494879999999999, 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134194 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770356+0000", 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770357+0000", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770359+0000", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0650000000000001e-06 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770365+0000", 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9349999999999999e-06 2024-09-16T22:17:26.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770370+0000", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8300000000000003e-06 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805771+0000", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035400993999999998 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805806+0000", 2024-09-16T22:17:26.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5466000000000002e-05 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805851+0000", 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5179000000000002e-05 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134195 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1030144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.988141079, 2024-09-16T22:17:26.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035569861000000001, 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134195 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770376+0000", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8299999999999998e-07 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770378+0000", 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8789999999999999e-06 2024-09-16T22:17:26.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770384+0000", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7030000000000003e-06 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770388+0000", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7659999999999998e-06 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805873+0000", 2024-09-16T22:17:26.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035484276000000002 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805909+0000", 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6155999999999997e-05 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805946+0000", 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6698e-05 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:134197 3.4 3:2725dfa6:::rbd_data.11cd14846e35.000000000000001d:26 [read 1034240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.988122733, 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035639397000000003, 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134197 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:26.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770394+0000", 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5400000000000002e-07 2024-09-16T22:17:26.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770396+0000", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6619999999999999e-06 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770401+0000", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4140000000000002e-06 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.770406+0000", 2024-09-16T22:17:26.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3599999999999998e-06 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805959+0000", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035553478999999999 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.805995+0000", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6214000000000003e-05 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.806033+0000", 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7914000000000003e-05 2024-09-16T22:17:26.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:26.859 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:26.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4856 writesame 0x973c52e thru 0x9741ae7 data_size 0x1(0x55ba bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4857 write 0xa291eeb thru 0xa299721 (0x7837 bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4858 punch from 0x82931a2 to 0x82961f5, (0x3053 bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4860 write 0xa523e85 thru 0xa52acfa (0x6e76 bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4862 writesame 0xc418825 thru 0xc420ae2 data_size 0x5(0x82be bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4863 read 0x171d571 thru 0x17227b0 (0x5240 bytes) 2024-09-16T22:17:26.862 INFO:teuthology.orchestra.run.smithi031.stdout:4864 trunc from 0xc420ae3 to 0x3ac9c4d 2024-09-16T22:17:26.862 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:26.861+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:26.863 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:26.863 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:17:26.863 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:26.864 INFO:teuthology.orchestra.run.smithi031.stdout:4865 write 0x547c105 thru 0x548613c (0xa038 bytes) 2024-09-16T22:17:26.873 INFO:teuthology.orchestra.run.smithi031.stdout:4866 trunc from 0x548613d to 0x1b89be4 2024-09-16T22:17:26.960 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:26.962+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:26.963 INFO:teuthology.orchestra.run.smithi031.stdout:4867 read 0x44ed48 thru 0x45cfcb (0xe284 bytes) 2024-09-16T22:17:26.971 INFO:teuthology.orchestra.run.smithi031.stdout:4868 write 0x35b76b8 thru 0x35c5ce6 (0xe62f bytes) 2024-09-16T22:17:26.992 INFO:teuthology.orchestra.run.smithi031.stdout:4872 write 0xa37faca thru 0xa38616e (0x66a5 bytes) 2024-09-16T22:17:27.000 INFO:teuthology.orchestra.run.smithi031.stdout:4873 writesame 0x2a69c2e thru 0x2a6d64a data_size 0x9(0x3a1d bytes) 2024-09-16T22:17:27.006 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:27.006 INFO:teuthology.orchestra.run.smithi031.stdout:4875 read 0x9f66cf7 thru 0x9f69417 (0x2721 bytes) 2024-09-16T22:17:27.009 INFO:teuthology.orchestra.run.smithi031.stdout:4876 trunc from 0xa38616f to 0xcfc5d24 2024-09-16T22:17:27.014 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:27.014 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:27.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:27.024 INFO:teuthology.orchestra.run.smithi031.stdout:4878 trunc from 0xcfc5d24 to 0x3eb979a 2024-09-16T22:17:27.043 INFO:teuthology.orchestra.run.smithi031.stdout:4879 trunc from 0x3eb979a to 0x6c7e18 2024-09-16T22:17:27.064 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:27.066+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:27.069 INFO:teuthology.orchestra.run.smithi031.stdout:4880 trunc from 0x6c7e18 to 0x940a32a 2024-09-16T22:17:27.075 INFO:teuthology.orchestra.run.smithi031.stdout:4881 writesame 0x4e33506 thru 0x4e3e8a7 data_size 0x1(0xb3a2 bytes) 2024-09-16T22:17:27.090 INFO:teuthology.orchestra.run.smithi031.stdout:4883 read 0x45c68ee thru 0x45d4dee (0xe501 bytes) 2024-09-16T22:17:27.091 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:27.091 INFO:teuthology.orchestra.run.smithi031.stdout:4886 punch from 0x2de3389 to 0x2de51ca, (0x1e41 bytes) 2024-09-16T22:17:27.099 INFO:teuthology.orchestra.run.smithi031.stdout:4887 write 0xbaca5db thru 0xbacdeb3 (0x38d9 bytes) 2024-09-16T22:17:27.106 INFO:teuthology.orchestra.run.smithi031.stdout:4888 write 0x916fc38 thru 0x917abc0 (0xaf89 bytes) 2024-09-16T22:17:27.115 INFO:teuthology.orchestra.run.smithi031.stdout:4889 write 0xa99a358 thru 0xa99b685 (0x132e bytes) 2024-09-16T22:17:27.118 INFO:teuthology.orchestra.run.smithi031.stdout:4890 read 0x1d65f36 thru 0x1d669e1 (0xaac bytes) 2024-09-16T22:17:27.118 INFO:teuthology.orchestra.run.smithi031.stdout:4891 read 0x7803209 thru 0x78103dc (0xd1d4 bytes) 2024-09-16T22:17:27.128 INFO:teuthology.orchestra.run.smithi031.stdout:4893 write 0xe7f0f97 thru 0xe7f4d24 (0x3d8e bytes) 2024-09-16T22:17:27.132 INFO:teuthology.orchestra.run.smithi031.stdout:4894 writesame 0xae08ab8 thru 0xae0aa57 data_size 0x2(0x1fa0 bytes) 2024-09-16T22:17:27.136 INFO:teuthology.orchestra.run.smithi031.stdout:4895 read 0xc0e3f3b thru 0xc0e51c6 (0x128c bytes) 2024-09-16T22:17:27.136 INFO:teuthology.orchestra.run.smithi031.stdout:4897 writesame 0x538158a thru 0x538683b data_size 0x1d(0x52b2 bytes) 2024-09-16T22:17:27.140 INFO:teuthology.orchestra.run.smithi031.stdout:4898 read 0xb445b5d thru 0xb447db3 (0x2257 bytes) 2024-09-16T22:17:27.140 INFO:teuthology.orchestra.run.smithi031.stdout:4900 write 0xa9f5c6f thru 0xaa0023e (0xa5d0 bytes) 2024-09-16T22:17:27.146 INFO:teuthology.orchestra.run.smithi031.stdout:4901 read 0x3546e9b thru 0x354c5ab (0x5711 bytes) 2024-09-16T22:17:27.146 INFO:teuthology.orchestra.run.smithi031.stdout:4903 write 0x986b4c8 thru 0x986ff42 (0x4a7b bytes) 2024-09-16T22:17:27.154 INFO:teuthology.orchestra.run.smithi031.stdout:4904 trunc from 0xe7f4d25 to 0x8d7f21f 2024-09-16T22:17:27.162 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:27.164+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:27.166 INFO:teuthology.orchestra.run.smithi031.stdout:4905 writesame 0x3c59337 thru 0x3c63817 data_size 0x1(0xa4e1 bytes) 2024-09-16T22:17:27.173 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:27.173 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:27.173 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:27.173 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:27.174 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:27.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:27.184 INFO:teuthology.orchestra.run.smithi031.stdout:4906 read 0x17351aa thru 0x17435cd (0xe424 bytes) 2024-09-16T22:17:27.185 INFO:teuthology.orchestra.run.smithi031.stdout:4907 punch from 0x87cf1b3 to 0x87d63cc, (0x7219 bytes) 2024-09-16T22:17:27.185 INFO:teuthology.orchestra.run.smithi031.stdout:4908 trunc from 0x8d7f21f to 0x82cda8a 2024-09-16T22:17:27.200 INFO:teuthology.orchestra.run.smithi031.stdout:4909 writesame 0xa12ed74 thru 0xa13d269 data_size 0x3(0xe4f6 bytes) 2024-09-16T22:17:27.200 INFO:teuthology.orchestra.run.smithi031.stdout:4910 writesame 0x116811f thru 0x116b6be data_size 0x1a(0x35a0 bytes) 2024-09-16T22:17:27.202 INFO:teuthology.orchestra.run.smithi031.stdout:4911 punch from 0xf310b to 0xfe189, (0xb07e bytes) 2024-09-16T22:17:27.202 INFO:teuthology.orchestra.run.smithi031.stdout:4912 read 0xe29cb0 thru 0xe2ef4b (0x529c bytes) 2024-09-16T22:17:27.202 INFO:teuthology.orchestra.run.smithi031.stdout:4913 punch from 0x38f7dc to 0x391ec4, (0x26e8 bytes) 2024-09-16T22:17:27.210 INFO:teuthology.orchestra.run.smithi031.stdout:4914 write 0xc11f578 thru 0xc12d3f6 (0xde7f bytes) 2024-09-16T22:17:27.221 INFO:teuthology.orchestra.run.smithi031.stdout:4915 read 0xbc13233 thru 0xbc156e3 (0x24b1 bytes) 2024-09-16T22:17:27.231 INFO:teuthology.orchestra.run.smithi031.stdout:4916 trunc from 0xc12d3f7 to 0x3535d1b 2024-09-16T22:17:27.249 INFO:teuthology.orchestra.run.smithi031.stdout:4917 writesame 0x38296e6 thru 0x38332ed data_size 0x4(0x9c08 bytes) 2024-09-16T22:17:27.250 INFO:teuthology.orchestra.run.smithi031.stdout:4918 trunc from 0x38332ee to 0x39dbf1a 2024-09-16T22:17:27.256 INFO:teuthology.orchestra.run.smithi031.stdout:4919 write 0x23d5d61 thru 0x23e3155 (0xd3f5 bytes) 2024-09-16T22:17:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:4920 clone 40 order 25 su 2097152 sc 2 2024-09-16T22:17:27.263 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:27.264+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:27.332 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:27.332 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:27.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:27.332 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:27.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557266952999999, 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:17:27.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557246794000001, 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:17:27.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:17:27.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557227752999999, 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:17:27.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:17:27.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557192346000001, 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557173184, 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:17:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:17:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:17:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557153902, 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557135204000001, 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557117468000001, 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:17:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:17:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557099837999999, 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:17:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:17:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557080619000001, 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:17:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:17:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557063519, 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:17:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557044782999998, 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:17:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:17:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557020936000001, 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:17:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:17:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.557001958000001, 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.556982842, 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:17:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.556952589000002, 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:17:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.556931788, 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:17:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:17:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.531355035000001, 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:17:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.531337456999999, 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.530319893000001, 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:17:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:17:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:27.425 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone39 2024-09-16T22:17:27.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:27.427 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:27.427+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-16T22:17:27.452 INFO:teuthology.orchestra.run.smithi031.stdout:4922 write 0xf670b4 thru 0xf671cf (0x11c bytes) 2024-09-16T22:17:27.482 INFO:teuthology.orchestra.run.smithi031.stdout:4924 write 0x19990f thru 0x1a7c08 (0xe2fa bytes) 2024-09-16T22:17:27.526 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:27.528+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:4926 writesame 0x4f597ce thru 0x4f59dc0 data_size 0x1(0x5f3 bytes) 2024-09-16T22:17:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:4928 read 0x34e9491 thru 0x34f7a30 (0xe5a0 bytes) 2024-09-16T22:17:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:4929 write 0x6dc3e40 thru 0x6dd0a83 (0xcc44 bytes) 2024-09-16T22:17:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:4930 read 0x455ce25 thru 0x45674dc (0xa6b8 bytes) 2024-09-16T22:17:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:4931 read 0x618b321 thru 0x6196e83 (0xbb63 bytes) 2024-09-16T22:17:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:4932 read 0x62fcab3 thru 0x63079b0 (0xaefe bytes) 2024-09-16T22:17:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:4933 read 0x1269e38 thru 0x126a1ec (0x3b5 bytes) 2024-09-16T22:17:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:4934 punch from 0x5ae8483 to 0x5af1fe6, (0x9b63 bytes) 2024-09-16T22:17:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:4936 read 0x46137f4 thru 0x461eaf6 (0xb303 bytes) 2024-09-16T22:17:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:4937 writesame 0x6a4fff thru 0x6aba3f data_size 0x1(0x6a41 bytes) 2024-09-16T22:17:27.549 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:27.549 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:27.549 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:27.550 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:27.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:27.560 INFO:teuthology.orchestra.run.smithi031.stdout:4938 trunc from 0x6dd0a84 to 0xa735ffa 2024-09-16T22:17:27.565 INFO:teuthology.orchestra.run.smithi031.stdout:4939 writesame 0x399c65c thru 0x399e7c2 data_size 0x1(0x2167 bytes) 2024-09-16T22:17:27.570 INFO:teuthology.orchestra.run.smithi031.stdout:4940 write 0x7ebb66e thru 0x7eca684 (0xf017 bytes) 2024-09-16T22:17:27.573 INFO:teuthology.orchestra.run.smithi031.stdout:4941 read 0x9fd67b1 thru 0x9fdf08e (0x88de bytes) 2024-09-16T22:17:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:4944 write 0xe0577da thru 0xe0639ca (0xc1f1 bytes) 2024-09-16T22:17:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:4945 read 0x85e74b2 thru 0x85ebc94 (0x47e3 bytes) 2024-09-16T22:17:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:4947 write 0x15df310 thru 0x15eeb06 (0xf7f7 bytes) 2024-09-16T22:17:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:4948 punch from 0x97d15cb to 0x97d4b24, (0x3559 bytes) 2024-09-16T22:17:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:4950 punch from 0x9ce6a5f to 0x9cf3c42, (0xd1e3 bytes) 2024-09-16T22:17:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:4951 write 0xa6cd9a9 thru 0xa6d66e3 (0x8d3b bytes) 2024-09-16T22:17:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:4952 read 0x638460d thru 0x63910c4 (0xcab8 bytes) 2024-09-16T22:17:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:4953 punch from 0x1c671bb to 0x1c762b3, (0xf0f8 bytes) 2024-09-16T22:17:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:4954 trunc from 0xe0639cb to 0x995ee5e 2024-09-16T22:17:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:4955 read 0x25abc9 thru 0x2685b2 (0xd9ea bytes) 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:4957 punch from 0x251b4d8 to 0x2523662, (0x818a bytes) 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:4960 read 0x32c74b3 thru 0x32cc81c (0x536a bytes) 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:4963 read 0x68c3e24 thru 0x68ccc06 (0x8de3 bytes) 2024-09-16T22:17:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:4964 read 0x9d20a9 thru 0x9d30b0 (0x1008 bytes) 2024-09-16T22:17:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:4965 read 0x18dfc55 thru 0x18ea6ca (0xaa76 bytes) 2024-09-16T22:17:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:4966 writesame 0x455ccc4 thru 0x45678ec data_size 0x53(0xac29 bytes) 2024-09-16T22:17:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:4967 writesame 0xd80128d thru 0xd806c57 data_size 0x1(0x59cb bytes) 2024-09-16T22:17:27.783 INFO:teuthology.orchestra.run.smithi031.stdout:4968 writesame 0x8c1dc92 thru 0x8c2da74 data_size 0x15(0xfde3 bytes) 2024-09-16T22:17:27.784 INFO:teuthology.orchestra.run.smithi031.stdout:4969 clone 41 order 18 su 16384 sc 13 2024-09-16T22:17:27.785 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:27.785 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:27.785 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:27.785 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:27.785 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:27.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:27.786 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:27.786+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:27.885 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:27.887+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.873317956999999, 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115259 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.852794217, 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043168221, 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115259 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082587+0000", 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087294+0000", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047076490000000004 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087309+0000", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4551e-05 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087318+0000", 2024-09-16T22:17:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087372+0000", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4184999999999997e-05 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087492+0000", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119762 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087887+0000", 2024-09-16T22:17:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039564899999999998 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.124754+0000", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036866492000000001 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125690+0000", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093677799999999998 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125726+0000", 2024-09-16T22:17:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5160999999999998e-05 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125752+0000", 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6177e-05 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131065 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 884736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.215053636, 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035774268999999997, 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131065 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-07 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720328+0000", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0169999999999998e-06 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720332+0000", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720334+0000", 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.542e-06 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.755899+0000", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035564506000000003 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756010+0000", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111073 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756098+0000", 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000001e-05 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131066 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 933888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.215040218, 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035857077000000001, 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131066 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.46e-07 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720339+0000", 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0580000000000001e-06 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720343+0000", 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720346+0000", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4360000000000001e-06 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756115+0000", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035768988000000002 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756155+0000", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0742999999999999e-05 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756195+0000", 2024-09-16T22:17:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9270999999999997e-05 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131067 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 983040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.215029058000001, 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036032743999999998, 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131067 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720350+0000", 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720353+0000", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720356+0000", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5289999999999998e-06 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756226+0000", 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035870446 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756303+0000", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6602999999999998e-05 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756382+0000", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8951999999999999e-05 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131068 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1032192~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.215019187999999, 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036117121000000002, 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131068 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720360+0000", 2024-09-16T22:17:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720363+0000", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720365+0000", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4719999999999998e-06 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756397+0000", 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036031615000000003 2024-09-16T22:17:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756437+0000", 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756476+0000", 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8804999999999999e-05 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131101 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1196032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.215009478000001, 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036281198000000001, 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131101 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720369+0000", 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720370+0000", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.023e-06 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720373+0000", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720375+0000", 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756508+0000", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036133086000000002 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756577+0000", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9148999999999998e-05 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756650+0000", 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2290000000000001e-05 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131102 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1245184~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214997807, 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036372546999999998, 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131102 2024-09-16T22:17:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720382+0000", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7120000000000001e-06 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720388+0000", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720398+0000", 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0433e-05 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756677+0000", 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036278794000000003 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756716+0000", 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9436e-05 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756753+0000", 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6282000000000003e-05 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131118 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1294336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214973353, 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036544986000000002, 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131118 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720405+0000", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720407+0000", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0159999999999998e-06 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720413+0000", 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6779999999999997e-06 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720417+0000", 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4909999999999999e-06 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756785+0000", 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036368075 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756867+0000", 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1853000000000004e-05 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756949+0000", 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2453000000000005e-05 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131171 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1056768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214853409, 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036546147000000001, 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131171 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720525+0000", 2024-09-16T22:17:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720526+0000", 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.029e-06 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720530+0000", 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-06 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720533+0000", 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756986+0000", 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036453754999999997 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757032+0000", 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5531999999999997e-05 2024-09-16T22:17:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757071+0000", 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8751000000000002e-05 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131172 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1105920~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214842045999999, 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036728516000000003, 2024-09-16T22:17:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131172 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720537+0000", 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720540+0000", 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720543+0000", 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757109+0000", 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036566047999999997 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757196+0000", 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7019000000000002e-05 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757264+0000", 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8374000000000006e-05 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131173 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1155072~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214831899, 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036824200000000001, 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131173 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-07 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720547+0000", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0979999999999999e-06 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720550+0000", 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.129e-06 2024-09-16T22:17:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720553+0000", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.492e-06 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757280+0000", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036726795 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757329+0000", 2024-09-16T22:17:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9703000000000002e-05 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757370+0000", 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0748999999999999e-05 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131174 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1204224~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214797721, 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036974039, 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131174 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720582+0000", 2024-09-16T22:17:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.531e-06 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720586+0000", 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720589+0000", 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8040000000000002e-06 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757402+0000", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036813564999999999 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757480+0000", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8103999999999996e-05 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757554+0000", 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3825999999999994e-05 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131175 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1253376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214785011, 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037074818000000002, 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131175 2024-09-16T22:17:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3000000000000002e-07 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720594+0000", 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720597+0000", 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720600+0000", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757586+0000", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036985393999999998 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757628+0000", 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2644e-05 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757668+0000", 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9307000000000002e-05 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131176 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1302528~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214774437000001, 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037249833000000003, 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131176 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720604+0000", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720605+0000", 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7199999999999997e-07 2024-09-16T22:17:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720608+0000", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.869e-06 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720610+0000", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757701+0000", 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037091164000000003 2024-09-16T22:17:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757780+0000", 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.852e-05 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757853+0000", 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3368999999999995e-05 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131177 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1089536~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214764558000001, 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037365571, 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131177 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720615+0000", 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.034e-06 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720617+0000", 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720620+0000", 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757885+0000", 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037265057999999997 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757935+0000", 2024-09-16T22:17:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0112999999999999e-05 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757979+0000", 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3584000000000001e-05 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131178 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1138688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214754660000001, 2024-09-16T22:17:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037553098, 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131178 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:28.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3099999999999999e-07 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720630+0000", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720633+0000", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2160000000000002e-06 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720636+0000", 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758017+0000", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037380913000000002 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758097+0000", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0029000000000002e-05 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758176+0000", 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9716999999999996e-05 2024-09-16T22:17:28.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131179 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1187840~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214738726, 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037658184999999997, 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131179 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720640+0000", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.037e-06 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720644+0000", 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0469999999999998e-06 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720646+0000", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5009999999999999e-06 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758204+0000", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037558084999999998 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758254+0000", 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9777000000000001e-05 2024-09-16T22:17:28.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758297+0000", 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3405000000000002e-05 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131180 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1236992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214728790000001, 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037835806, 2024-09-16T22:17:28.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131180 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720650+0000", 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031e-06 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720653+0000", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.892e-06 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720656+0000", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.339e-06 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758335+0000", 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037679556000000003 2024-09-16T22:17:28.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758409+0000", 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3980000000000007e-05 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758485+0000", 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5797999999999995e-05 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131181 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1286144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.214719291, 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037949063999999998, 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131181 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4299999999999999e-07 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720660+0000", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720663+0000", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720665+0000", 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-16T22:17:28.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758515+0000", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037850097999999999 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758564+0000", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8179000000000001e-05 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758608+0000", 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4088000000000002e-05 2024-09-16T22:17:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:28.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:28.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:28.028 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:28.029+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:28.128 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:28.130+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:28.175 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:28.175 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:28.175 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:28.175 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:28.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:28.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:28.229 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:28.231+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:28.292 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:28.292 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:28.292 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:28.292 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:28.292 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:28.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:28.344 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:28.346+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:28.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:17:28.380 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:28.381 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone40 from 0xd806c58 (overlap 0x39dbf1a) to 0x3412bfd 2024-09-16T22:17:28.402 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:28.402 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:28.419 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:28.421 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:28.421 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:28.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:28.431 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:28.432+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:28.449 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:28.469 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:28.483 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-16T22:17:28.489 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:28.524 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:28.524 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:28.532 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:28.533+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:28.557 INFO:teuthology.orchestra.run.smithi031.stdout:4971 read 0x8905b88 thru 0x890995b (0x3dd4 bytes) 2024-09-16T22:17:28.558 INFO:teuthology.orchestra.run.smithi031.stdout:4972 write 0xa3bc3bb thru 0xa3c256b (0x61b1 bytes) 2024-09-16T22:17:28.572 INFO:teuthology.orchestra.run.smithi031.stdout:4973 punch from 0x46fe26e to 0x46ff59b, (0x132d bytes) 2024-09-16T22:17:28.572 INFO:teuthology.orchestra.run.smithi031.stdout:4974 writesame 0xd06de78 thru 0xd0750a7 data_size 0x7e(0x7230 bytes) 2024-09-16T22:17:28.583 INFO:teuthology.orchestra.run.smithi031.stdout:4975 trunc from 0xd806c58 to 0xb4022d3 2024-09-16T22:17:28.584 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:28.584 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:28.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:28.628 INFO:teuthology.orchestra.run.smithi031.stdout:4978 trunc from 0xb4022d3 to 0x633f795 2024-09-16T22:17:28.665 INFO:teuthology.orchestra.run.smithi031.stdout:4983 punch from 0x4e78ac2 to 0x4e78ff0, (0x52e bytes) 2024-09-16T22:17:28.666 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:28.667+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:28.673 INFO:teuthology.orchestra.run.smithi031.stdout:4984 write 0x7da8d24 thru 0x7daf811 (0x6aee bytes) 2024-09-16T22:17:28.677 INFO:teuthology.orchestra.run.smithi031.stdout:4986 writesame 0x5173edc thru 0x5181a4c data_size 0x1(0xdb71 bytes) 2024-09-16T22:17:28.706 INFO:teuthology.orchestra.run.smithi031.stdout:4988 write 0x87e89f6 thru 0x87f88f2 (0xfefd bytes) 2024-09-16T22:17:28.717 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:28.728 INFO:teuthology.orchestra.run.smithi031.stdout:4990 write 0xb63ef29 thru 0xb644271 (0x5349 bytes) 2024-09-16T22:17:28.734 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:28.735+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:28.734 INFO:teuthology.orchestra.run.smithi031.stdout:4992 read 0x4e1cba6 thru 0x4e28a78 (0xbed3 bytes) 2024-09-16T22:17:28.735 INFO:teuthology.orchestra.run.smithi031.stdout:4993 writesame 0x971b24c thru 0x971d4b2 data_size 0x1(0x2267 bytes) 2024-09-16T22:17:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:4994 clone 42 order 25 su 16777216 sc 5 2024-09-16T22:17:28.764 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:28.764 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:28.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:28.834 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:28.836+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:28.921 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:28.921 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:28.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:28.935 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:28.937+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:29.036 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:29.038+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:29.137 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:29.139+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:29.152 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.153 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:29.238 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:29.240+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:29.309 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.309 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.316 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:17:29.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:29.364 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:29.366+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone41 from 0xb644272 (overlap 0x633f795) to 0x13d97ea 2024-09-16T22:17:29.440 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:29.442+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:29.463 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.464 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:29.561 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:29.563+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:29.642 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:29.644+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:29.670 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.670 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:29.743 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:29.744+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:29.843 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:29.845+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:29.855 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.855 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:29.944 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:29.945+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:29.984 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-16T22:17:29.986 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:29.986 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:29.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:30.044 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:30.046+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:30.055 INFO:teuthology.orchestra.run.smithi031.stdout:4996 read 0x23333c6 thru 0x234218b (0xedc6 bytes) 2024-09-16T22:17:30.057 INFO:teuthology.orchestra.run.smithi031.stdout:4997 write 0x25486af thru 0x2551657 (0x8fa9 bytes) 2024-09-16T22:17:30.121 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:30.121 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:30.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:30.174 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:30.176+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:30.208 INFO:teuthology.orchestra.run.smithi031.stdout:4998 punch from 0x730de73 to 0x731144f, (0x35dc bytes) 2024-09-16T22:17:30.208 INFO:teuthology.orchestra.run.smithi031.stdout:4999 read 0x3cef828 thru 0x3cfa50d (0xace6 bytes) 2024-09-16T22:17:30.209 INFO:teuthology.orchestra.run.smithi031.stdout:5000 read 0x293a192 thru 0x2946d48 (0xcbb7 bytes) 2024-09-16T22:17:30.210 INFO:teuthology.orchestra.run.smithi031.stdout:5002 read 0xa67f47f thru 0xa685b19 (0x669b bytes) 2024-09-16T22:17:30.210 INFO:teuthology.orchestra.run.smithi031.stdout:5003 write 0x2dd5d35 thru 0x2ddf3f2 (0x96be bytes) 2024-09-16T22:17:30.212 INFO:teuthology.orchestra.run.smithi031.stdout:5005 read 0x1af9d5a thru 0x1afc37c (0x2623 bytes) 2024-09-16T22:17:30.213 INFO:teuthology.orchestra.run.smithi031.stdout:5006 read 0xa7cb78 thru 0xa80725 (0x3bae bytes) 2024-09-16T22:17:30.213 INFO:teuthology.orchestra.run.smithi031.stdout:5007 read 0x51658d0 thru 0x516faf8 (0xa229 bytes) 2024-09-16T22:17:30.214 INFO:teuthology.orchestra.run.smithi031.stdout:5008 flatten 2024-09-16T22:17:30.246 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:30.248+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:30.302 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:30.303 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:30.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:30.357 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:30.359+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:30.448 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:30.449+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:30.464 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:30.465 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:30.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:30.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:30.582 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:30.583+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:30.650 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:30.652+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:30.676 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:30.676 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:30.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:30.715 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:30.715 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:30.733 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:30.749 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:30.750 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:30.750 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:30.752+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:30.766 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:30.766 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:30.774 INFO:teuthology.orchestra.run.smithi031.stdout:5009 trunc from 0xb644272 to 0xb0ae6dc 2024-09-16T22:17:30.780 INFO:teuthology.orchestra.run.smithi031.stdout:5010 read 0x1c72d97 thru 0x1c77544 (0x47ae bytes) 2024-09-16T22:17:30.784 INFO:teuthology.orchestra.run.smithi031.stdout:5011 trunc from 0xb0ae6dc to 0xdf99f5b 2024-09-16T22:17:30.788 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:30.788 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:30.789 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.790 INFO:teuthology.orchestra.run.smithi031.stdout:5014 read 0x2297a1c thru 0x229a4c9 (0x2aae bytes) 2024-09-16T22:17:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:5015 punch from 0x62be885 to 0x62ca397, (0xbb12 bytes) 2024-09-16T22:17:30.797 INFO:teuthology.orchestra.run.smithi031.stdout:5016 trunc from 0xdf99f5b to 0x910f722 2024-09-16T22:17:30.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:30.804 INFO:teuthology.orchestra.run.smithi031.stdout:5017 read 0x55aa7c1 thru 0x55b4ad0 (0xa310 bytes) 2024-09-16T22:17:30.804 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.805 INFO:teuthology.orchestra.run.smithi031.stdout:5022 read 0x73ad593 thru 0x73aff86 (0x29f4 bytes) 2024-09-16T22:17:30.805 INFO:teuthology.orchestra.run.smithi031.stdout:5024 write 0x64f7fb thru 0x65b76a (0xbf70 bytes) 2024-09-16T22:17:30.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:30.817 INFO:teuthology.orchestra.run.smithi031.stdout:5025 writesame 0xb70dd8b thru 0xb719460 data_size 0x57(0xb6d6 bytes) 2024-09-16T22:17:30.817 INFO:teuthology.orchestra.run.smithi031.stdout:5027 write 0x627840c thru 0x6287057 (0xec4c bytes) 2024-09-16T22:17:30.818 INFO:teuthology.orchestra.run.smithi031.stdout:5029 read 0x4e5dabb thru 0x4e66e82 (0x93c8 bytes) 2024-09-16T22:17:30.819 INFO:teuthology.orchestra.run.smithi031.stdout:5030 read 0x94a454d thru 0x94a4828 (0x2dc bytes) 2024-09-16T22:17:30.819 INFO:teuthology.orchestra.run.smithi031.stdout:5032 writesame 0x8c67b96 thru 0x8c76dc7 data_size 0x1d(0xf232 bytes) 2024-09-16T22:17:30.821 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.823 INFO:teuthology.orchestra.run.smithi031.stdout:5034 trunc from 0xb719461 to 0xa090583 2024-09-16T22:17:30.828 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:30.828 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:30.830 INFO:teuthology.orchestra.run.smithi031.stdout:5035 write 0x70c7d2 thru 0x710bf0 (0x441f bytes) 2024-09-16T22:17:30.832 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.832 INFO:teuthology.orchestra.run.smithi031.stdout:5037 read 0x979078f thru 0x979ad82 (0xa5f4 bytes) 2024-09-16T22:17:30.832 INFO:teuthology.orchestra.run.smithi031.stdout:5038 write 0x5c82cb5 thru 0x5c86dd8 (0x4124 bytes) 2024-09-16T22:17:30.834 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.834 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.834 INFO:teuthology.orchestra.run.smithi031.stdout:5042 read 0x4120d39 thru 0x41212c2 (0x58a bytes) 2024-09-16T22:17:30.836 INFO:teuthology.orchestra.run.smithi031.stdout:5043 punch from 0x13589a2 to 0x1361f6e, (0x95cc bytes) 2024-09-16T22:17:30.842 INFO:teuthology.orchestra.run.smithi031.stdout:5044 trunc from 0xa090583 to 0x59b920a 2024-09-16T22:17:30.849 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:30.850 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:30.851 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:30.853+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:30.853 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.859 INFO:teuthology.orchestra.run.smithi031.stdout:5046 trunc from 0x59b920a to 0xa5c1ff3 2024-09-16T22:17:30.865 INFO:teuthology.orchestra.run.smithi031.stdout:5047 read 0x57474ef thru 0x5752f62 (0xba74 bytes) 2024-09-16T22:17:30.867 INFO:teuthology.orchestra.run.smithi031.stdout:5048 write 0x9531fe5 thru 0x954108e (0xf0aa bytes) 2024-09-16T22:17:30.877 INFO:teuthology.orchestra.run.smithi031.stdout:5049 write 0xcaa376f thru 0xcaad5ad (0x9e3f bytes) 2024-09-16T22:17:30.881 INFO:teuthology.orchestra.run.smithi031.stdout:5050 punch from 0xda6955 to 0xdb1b78, (0xb223 bytes) 2024-09-16T22:17:30.881 INFO:teuthology.orchestra.run.smithi031.stdout:5051 read 0x2ca1b89 thru 0x2cae279 (0xc6f1 bytes) 2024-09-16T22:17:30.881 INFO:teuthology.orchestra.run.smithi031.stdout:5052 read 0x5432017 thru 0x5440847 (0xe831 bytes) 2024-09-16T22:17:30.882 INFO:teuthology.orchestra.run.smithi031.stdout:5053 write 0x474b9ba thru 0x475018c (0x47d3 bytes) 2024-09-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:5054 writesame 0x391970c thru 0x39276db data_size 0x8(0xdfd0 bytes) 2024-09-16T22:17:30.889 INFO:teuthology.orchestra.run.smithi031.stdout:5055 read 0xa526e0c thru 0xa529700 (0x28f5 bytes) 2024-09-16T22:17:30.900 INFO:teuthology.orchestra.run.smithi031.stdout:5056 trunc from 0xcaad5ae to 0x344631a 2024-09-16T22:17:30.909 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:30.909 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:30.909 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:30.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:30.920 INFO:teuthology.orchestra.run.smithi031.stdout:5057 punch from 0x22354d to 0x22ac9e, (0x7751 bytes) 2024-09-16T22:17:30.920 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:30.922 INFO:teuthology.orchestra.run.smithi031.stdout:5059 trunc from 0x344631a to 0xea8bad 2024-09-16T22:17:30.944 INFO:teuthology.orchestra.run.smithi031.stdout:5060 punch from 0xc94e85 to 0xc97922, (0x2a9d bytes) 2024-09-16T22:17:30.944 INFO:teuthology.orchestra.run.smithi031.stdout:5062 punch from 0x613cc2 to 0x61f9c5, (0xbd03 bytes) 2024-09-16T22:17:30.945 INFO:teuthology.orchestra.run.smithi031.stdout:5063 write 0x7e69ad thru 0x7e7676 (0xcca bytes) 2024-09-16T22:17:30.945 INFO:teuthology.orchestra.run.smithi031.stdout:5064 read 0x48d87e thru 0x49b9b1 (0xe134 bytes) 2024-09-16T22:17:30.956 INFO:teuthology.orchestra.run.smithi031.stdout:5065 write 0x442e690 thru 0x442ea81 (0x3f2 bytes) 2024-09-16T22:17:30.959 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:30.961+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:30.960 INFO:teuthology.orchestra.run.smithi031.stdout:5066 read 0x298dd3f thru 0x2992e5f (0x5121 bytes) 2024-09-16T22:17:30.960 INFO:teuthology.orchestra.run.smithi031.stdout:5069 read 0x4354bd9 thru 0x4358fb5 (0x43dd bytes) 2024-09-16T22:17:30.960 INFO:teuthology.orchestra.run.smithi031.stdout:5070 punch from 0x391e004 to 0x392017b, (0x2177 bytes) 2024-09-16T22:17:30.960 INFO:teuthology.orchestra.run.smithi031.stdout:5071 punch from 0x24cda2f to 0x24dd198, (0xf769 bytes) 2024-09-16T22:17:30.960 INFO:teuthology.orchestra.run.smithi031.stdout:5074 write 0x3822a0d thru 0x38326e2 (0xfcd6 bytes) 2024-09-16T22:17:30.970 INFO:teuthology.orchestra.run.smithi031.stdout:5075 writesame 0x5b1bef6 thru 0x5b298ee data_size 0x1(0xd9f9 bytes) 2024-09-16T22:17:30.981 INFO:teuthology.orchestra.run.smithi031.stdout:5076 writesame 0x8b3ac5a thru 0x8b400cf data_size 0x13(0x5476 bytes) 2024-09-16T22:17:30.983 INFO:teuthology.orchestra.run.smithi031.stdout:5077 writesame 0x46934af thru 0x4696513 data_size 0x1(0x3065 bytes) 2024-09-16T22:17:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:5078 write 0x70100ec thru 0x701e8b3 (0xe7c8 bytes) 2024-09-16T22:17:31.000 INFO:teuthology.orchestra.run.smithi031.stdout:5079 write 0x9d34388 thru 0x9d37764 (0x33dd bytes) 2024-09-16T22:17:31.001 INFO:teuthology.orchestra.run.smithi031.stdout:5080 write 0x55b0ff5 thru 0x55b1898 (0x8a4 bytes) 2024-09-16T22:17:31.002 INFO:teuthology.orchestra.run.smithi031.stdout:5081 punch from 0x99c39a1 to 0x99d3308, (0xf967 bytes) 2024-09-16T22:17:31.003 INFO:teuthology.orchestra.run.smithi031.stdout:5082 read 0x3cb2d9 thru 0x3cc741 (0x1469 bytes) 2024-09-16T22:17:31.003 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.003 INFO:teuthology.orchestra.run.smithi031.stdout:5084 writesame 0x47ecf52 thru 0x47f0088 data_size 0x1(0x3137 bytes) 2024-09-16T22:17:31.015 INFO:teuthology.orchestra.run.smithi031.stdout:5085 trunc from 0x9d37765 to 0x5a8b80f 2024-09-16T22:17:31.022 INFO:teuthology.orchestra.run.smithi031.stdout:5086 punch from 0x8f918b to 0x908ace, (0xf943 bytes) 2024-09-16T22:17:31.023 INFO:teuthology.orchestra.run.smithi031.stdout:5087 punch from 0xe6c050 to 0xe7ad49, (0xecf9 bytes) 2024-09-16T22:17:31.023 INFO:teuthology.orchestra.run.smithi031.stdout:5089 write 0x7a16a7 thru 0x7a70b3 (0x5a0d bytes) 2024-09-16T22:17:31.033 INFO:teuthology.orchestra.run.smithi031.stdout:5093 trunc from 0x5a8b80f to 0xcce7ed8 2024-09-16T22:17:31.039 INFO:teuthology.orchestra.run.smithi031.stdout:5094 write 0x6897199 thru 0x68a2c60 (0xbac8 bytes) 2024-09-16T22:17:31.042 INFO:teuthology.orchestra.run.smithi031.stdout:5096 read 0xbf95197 thru 0xbf977b6 (0x2620 bytes) 2024-09-16T22:17:31.042 INFO:teuthology.orchestra.run.smithi031.stdout:5097 read 0x14e4f85 thru 0x14f3980 (0xe9fc bytes) 2024-09-16T22:17:31.042 INFO:teuthology.orchestra.run.smithi031.stdout:5098 read 0x688751d thru 0x68900ad (0x8b91 bytes) 2024-09-16T22:17:31.046 INFO:teuthology.orchestra.run.smithi031.stdout:5099 trunc from 0xcce7ed8 to 0xa2fbb40 2024-09-16T22:17:31.051 INFO:teuthology.orchestra.run.smithi031.stdout:5100 read 0x8065865 thru 0x806b747 (0x5ee3 bytes) 2024-09-16T22:17:31.053 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:31.055+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:31.060 INFO:teuthology.orchestra.run.smithi031.stdout:5103 writesame 0xc1b5a7e thru 0xc1bb27e data_size 0x1(0x5801 bytes) 2024-09-16T22:17:31.060 INFO:teuthology.orchestra.run.smithi031.stdout:5104 trunc from 0xc1bb27f to 0xbf4b72e 2024-09-16T22:17:31.064 INFO:teuthology.orchestra.run.smithi031.stdout:5105 write 0x2594b7f thru 0x25a06cb (0xbb4d bytes) 2024-09-16T22:17:31.066 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.066 INFO:teuthology.orchestra.run.smithi031.stdout:5107 read 0x9e925ff thru 0x9e96b22 (0x4524 bytes) 2024-09-16T22:17:31.066 INFO:teuthology.orchestra.run.smithi031.stdout:5108 read 0x22d6fad thru 0x22da5a8 (0x35fc bytes) 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:31.067 INFO:teuthology.orchestra.run.smithi031.stdout:5110 write 0xa5ce2e0 thru 0xa5d572f (0x7450 bytes) 2024-09-16T22:17:31.070 INFO:teuthology.orchestra.run.smithi031.stdout:5111 writesame 0x21972fc thru 0x21a3cbb data_size 0x30(0xc9c0 bytes) 2024-09-16T22:17:31.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:5113 write 0xc46ea6a thru 0xc4784c2 (0x9a59 bytes) 2024-09-16T22:17:31.080 INFO:teuthology.orchestra.run.smithi031.stdout:5114 read 0x552a445 thru 0x5536da6 (0xc962 bytes) 2024-09-16T22:17:31.080 INFO:teuthology.orchestra.run.smithi031.stdout:5115 writesame 0x5346b31 thru 0x534d3fa data_size 0x33(0x68ca bytes) 2024-09-16T22:17:31.082 INFO:teuthology.orchestra.run.smithi031.stdout:5116 read 0xafc6b7a thru 0xafd1218 (0xa69f bytes) 2024-09-16T22:17:31.082 INFO:teuthology.orchestra.run.smithi031.stdout:5117 write 0x31335d3 thru 0x3141050 (0xda7e bytes) 2024-09-16T22:17:31.084 INFO:teuthology.orchestra.run.smithi031.stdout:5118 punch from 0x11f5339 to 0x1204f15, (0xfbdc bytes) 2024-09-16T22:17:31.084 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.095 INFO:teuthology.orchestra.run.smithi031.stdout:5120 trunc from 0xc4784c3 to 0x1e93337 2024-09-16T22:17:31.112 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.112 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:5127 writesame 0xe56ad3f thru 0xe5795e0 data_size 0xb(0xe8a2 bytes) 2024-09-16T22:17:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:5131 read 0xe4e3fc thru 0xe59be0 (0xb7e5 bytes) 2024-09-16T22:17:31.133 INFO:teuthology.orchestra.run.smithi031.stdout:5132 write 0xbbed59a thru 0xbbfab50 (0xd5b7 bytes) 2024-09-16T22:17:31.137 INFO:teuthology.orchestra.run.smithi031.stdout:5133 read 0x73d52a3 thru 0x73d6f9d (0x1cfb bytes) 2024-09-16T22:17:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:5134 writesame 0xeab8502 thru 0xeaba17d data_size 0x2(0x1c7c bytes) 2024-09-16T22:17:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:5135 write 0xcee1725 thru 0xceee8fc (0xd1d8 bytes) 2024-09-16T22:17:31.146 INFO:teuthology.orchestra.run.smithi031.stdout:5136 write 0xc35df41 thru 0xc36331a (0x53da bytes) 2024-09-16T22:17:31.148 INFO:teuthology.orchestra.run.smithi031.stdout:5137 read 0x791ee6d thru 0x79278a5 (0x8a39 bytes) 2024-09-16T22:17:31.148 INFO:teuthology.orchestra.run.smithi031.stdout:5140 trunc from 0xeaba17e to 0xe1d6605 2024-09-16T22:17:31.153 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:31.155+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:31.158 INFO:teuthology.orchestra.run.smithi031.stdout:5142 trunc from 0xe1d6605 to 0xb8fb844 2024-09-16T22:17:31.171 INFO:teuthology.orchestra.run.smithi031.stdout:5143 punch from 0xd2d77d to 0xd3a442, (0xccc5 bytes) 2024-09-16T22:17:31.171 INFO:teuthology.orchestra.run.smithi031.stdout:5144 punch from 0x159ee2 to 0x15c7c6, (0x28e4 bytes) 2024-09-16T22:17:31.171 INFO:teuthology.orchestra.run.smithi031.stdout:5145 read 0x929236b thru 0x9292b36 (0x7cc bytes) 2024-09-16T22:17:31.172 INFO:teuthology.orchestra.run.smithi031.stdout:5146 trunc from 0xb8fb844 to 0xa9b7c25 2024-09-16T22:17:31.186 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:31.186 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:31.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:31.186 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:31.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411160918, 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:17:31.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411140758999998, 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:31.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:17:31.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411121718, 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411086310999998, 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:17:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:17:31.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411067149000001, 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411047867000001, 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411029168999999, 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:17:31.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.411011432999999, 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410993803, 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410974584000002, 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410957484000001, 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:17:31.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410938748, 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:17:31.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410914901000002, 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:17:31.238 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410895922999998, 2024-09-16T22:17:31.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:17:31.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410876807000001, 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:31.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:17:31.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:17:31.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410846553999999, 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.251 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:17:31.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:17:31.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.410825753000001, 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:17:31.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:17:31.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.385249000000002, 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.385231422, 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:17:31.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.384213857999999, 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:31.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:17:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:17:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:31.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5147 trunc from 0xa9b7c25 to 0x4157885 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5148 read 0x2863dec thru 0x286ebe2 (0xadf7 bytes) 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5149 write 0xa3e4d98 thru 0xa3f121a (0xc483 bytes) 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5153 writesame 0x75c2a74 thru 0x75c7550 data_size 0x1(0x4add bytes) 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5154 writesame 0x9e187ae thru 0x9e1e264 data_size 0x1(0x5ab7 bytes) 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:5155 write 0xe043ccb thru 0xe045be8 (0x1f1e bytes) 2024-09-16T22:17:31.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5157 read 0x162d2ea thru 0x1630518 (0x322f bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5158 write 0x6d02d38 thru 0x6d0958c (0x6855 bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5159 trunc from 0xe045be9 to 0xb6fd9f4 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5163 read 0xb6cc14c thru 0xb6db133 (0xefe8 bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5165 read 0x110de6d thru 0x1111fef (0x4183 bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5167 write 0x4fcc418 thru 0x4fd6645 (0xa22e bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5168 write 0xaa72c76 thru 0xaa73c33 (0xfbe bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:5171 punch from 0xb6cf039 to 0xb6d6f5d, (0x7f24 bytes) 2024-09-16T22:17:31.276 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.277 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:31.275+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:31.277 INFO:teuthology.orchestra.run.smithi031.stdout:5174 write 0xd129b4d thru 0xd1302db (0x678f bytes) 2024-09-16T22:17:31.281 INFO:teuthology.orchestra.run.smithi031.stdout:5179 write 0x85f416 thru 0x86a067 (0xac52 bytes) 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:5180 punch from 0x2d8d2a7 to 0x2d93397, (0x60f0 bytes) 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:5182 punch from 0xb3b60fa to 0xb3c329c, (0xd1a2 bytes) 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:5183 punch from 0x1aae35a to 0x1ab2b90, (0x4836 bytes) 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.284 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.289 INFO:teuthology.orchestra.run.smithi031.stdout:5187 trunc from 0xd1302dc to 0x809cbc5 2024-09-16T22:17:31.302 INFO:teuthology.orchestra.run.smithi031.stdout:5188 write 0x599429e thru 0x59a0ae2 (0xc845 bytes) 2024-09-16T22:17:31.304 INFO:teuthology.orchestra.run.smithi031.stdout:5189 writesame 0x32d82bf thru 0x32e5579 data_size 0x1(0xd2bb bytes) 2024-09-16T22:17:31.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:17:31.350 INFO:teuthology.orchestra.run.smithi031.stdout:5190 write 0xaf58223 thru 0xaf5d7c3 (0x55a1 bytes) 2024-09-16T22:17:31.354 INFO:teuthology.orchestra.run.smithi031.stdout:5191 punch from 0x9ea7284 to 0x9eb1220, (0x9f9c bytes) 2024-09-16T22:17:31.355 INFO:teuthology.orchestra.run.smithi031.stdout:5192 write 0x81a9b9e thru 0x81ae773 (0x4bd6 bytes) 2024-09-16T22:17:31.357 INFO:teuthology.orchestra.run.smithi031.stdout:5193 read 0x78ff3a2 thru 0x7909f0a (0xab69 bytes) 2024-09-16T22:17:31.357 INFO:teuthology.orchestra.run.smithi031.stdout:5197 write 0x68e2ea9 thru 0x68e4b38 (0x1c90 bytes) 2024-09-16T22:17:31.359 INFO:teuthology.orchestra.run.smithi031.stdout:5198 read 0x47c7749 thru 0x47d73e9 (0xfca1 bytes) 2024-09-16T22:17:31.359 INFO:teuthology.orchestra.run.smithi031.stdout:5199 read 0x56ab3a7 thru 0x56ac61d (0x1277 bytes) 2024-09-16T22:17:31.374 INFO:teuthology.orchestra.run.smithi031.stdout:5200 trunc from 0xaf5d7c4 to 0x5c5f37 2024-09-16T22:17:31.374 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:31.376+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:31.398 INFO:teuthology.orchestra.run.smithi031.stdout:5201 read 0x2ee611 thru 0x2f478b (0x617b bytes) 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185221 3.f 3:f70f30da:::rbd_object_map.11cded7d7aa9:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.410277+0000", 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0010653329999999999, 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001096103, 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for sub ops", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185221 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410277+0000", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410277+0000", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410279+0000", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410292+0000", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3315999999999999e-05 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410314+0000", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1522000000000002e-05 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410326+0000", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2362e-05 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410394+0000", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.847e-05 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410468+0000", 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4093999999999999e-05 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.410652+0000", 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000183241 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-16T22:17:31.413 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:31.414 INFO:teuthology.orchestra.run.smithi031.stdout:5202 write 0x6e4cfd5 thru 0x6e50645 (0x3671 bytes) 2024-09-16T22:17:31.417 INFO:teuthology.orchestra.run.smithi031.stdout:5205 read 0x52e53a9 thru 0x52ee736 (0x938e bytes) 2024-09-16T22:17:31.417 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.417 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.417 INFO:teuthology.orchestra.run.smithi031.stdout:5208 read 0x279fa98 thru 0x27aecc9 (0xf232 bytes) 2024-09-16T22:17:31.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:31.430 INFO:teuthology.orchestra.run.smithi031.stdout:5210 writesame 0xc4ecdb0 thru 0xc4f2898 data_size 0x11(0x5ae9 bytes) 2024-09-16T22:17:31.430 INFO:teuthology.orchestra.run.smithi031.stdout:5211 write 0xa7dd38d thru 0xa7eba1a (0xe68e bytes) 2024-09-16T22:17:31.439 INFO:teuthology.orchestra.run.smithi031.stdout:5212 trunc from 0xc4f2899 to 0x8fb648a 2024-09-16T22:17:31.458 INFO:teuthology.orchestra.run.smithi031.stdout:5213 writesame 0x9983648 thru 0x99870ea data_size 0x1(0x3aa3 bytes) 2024-09-16T22:17:31.458 INFO:teuthology.orchestra.run.smithi031.stdout:5214 write 0x7931e45 thru 0x793455c (0x2718 bytes) 2024-09-16T22:17:31.461 INFO:teuthology.orchestra.run.smithi031.stdout:5215 read 0xad7a37 thru 0xae5546 (0xdb10 bytes) 2024-09-16T22:17:31.462 INFO:teuthology.orchestra.run.smithi031.stdout:5216 punch from 0x5543733 to 0x554ad28, (0x75f5 bytes) 2024-09-16T22:17:31.462 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:5219 write 0xe5c87df thru 0xe5cf385 (0x6ba7 bytes) 2024-09-16T22:17:31.475 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:31.477+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:31.477 INFO:teuthology.orchestra.run.smithi031.stdout:5220 read 0x9e405d0 thru 0x9e4b58e (0xafbf bytes) 2024-09-16T22:17:31.477 INFO:teuthology.orchestra.run.smithi031.stdout:5222 write 0x4a3766b thru 0x4a3cd30 (0x56c6 bytes) 2024-09-16T22:17:31.480 INFO:teuthology.orchestra.run.smithi031.stdout:5223 read 0xe3c7fe8 thru 0xe3d66ce (0xe6e7 bytes) 2024-09-16T22:17:31.481 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.481 INFO:teuthology.orchestra.run.smithi031.stdout:5226 write 0x18b93a0 thru 0x18bcf6d (0x3bce bytes) 2024-09-16T22:17:31.482 INFO:teuthology.orchestra.run.smithi031.stdout:5227 write 0x448e727 thru 0x449e62a (0xff04 bytes) 2024-09-16T22:17:31.484 INFO:teuthology.orchestra.run.smithi031.stdout:5228 read 0xa5349e5 thru 0xa53676e (0x1d8a bytes) 2024-09-16T22:17:31.484 INFO:teuthology.orchestra.run.smithi031.stdout:5229 read 0xb64724a thru 0xb656508 (0xf2bf bytes) 2024-09-16T22:17:31.485 INFO:teuthology.orchestra.run.smithi031.stdout:5232 write 0xb9a75df thru 0xb9aec14 (0x7636 bytes) 2024-09-16T22:17:31.488 INFO:teuthology.orchestra.run.smithi031.stdout:5233 write 0xa960765 thru 0xa960f64 (0x800 bytes) 2024-09-16T22:17:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.501 INFO:teuthology.orchestra.run.smithi031.stdout:5235 trunc from 0xe5cf386 to 0x607d08b 2024-09-16T22:17:31.514 INFO:teuthology.orchestra.run.smithi031.stdout:5237 punch from 0x80f7a to 0x8ab12, (0x9b98 bytes) 2024-09-16T22:17:31.514 INFO:teuthology.orchestra.run.smithi031.stdout:5238 read 0x51757f4 thru 0x5181272 (0xba7f bytes) 2024-09-16T22:17:31.521 INFO:teuthology.orchestra.run.smithi031.stdout:5239 trunc from 0x607d08b to 0xbb01b31 2024-09-16T22:17:31.527 INFO:teuthology.orchestra.run.smithi031.stdout:5241 read 0xb0c9734 thru 0xb0cf2fe (0x5bcb bytes) 2024-09-16T22:17:31.527 INFO:teuthology.orchestra.run.smithi031.stdout:5242 punch from 0x11c44a to 0x12a60e, (0xe1c4 bytes) 2024-09-16T22:17:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:5243 write 0x70bc80c thru 0x70c2fa8 (0x679d bytes) 2024-09-16T22:17:31.529 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.529 INFO:teuthology.orchestra.run.smithi031.stdout:5245 writesame 0x7985ab2 thru 0x7988a56 data_size 0x1(0x2fa5 bytes) 2024-09-16T22:17:31.534 INFO:teuthology.orchestra.run.smithi031.stdout:5246 write 0x9b47817 thru 0x9b52c4b (0xb435 bytes) 2024-09-16T22:17:31.536 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.536 INFO:teuthology.orchestra.run.smithi031.stdout:5248 write 0xa7517ea thru 0xa759cdb (0x84f2 bytes) 2024-09-16T22:17:31.541 INFO:teuthology.orchestra.run.smithi031.stdout:5250 trunc from 0xbb01b31 to 0xc472278 2024-09-16T22:17:31.546 INFO:teuthology.orchestra.run.smithi031.stdout:5251 punch from 0x6801fa2 to 0x6802b50, (0xbae bytes) 2024-09-16T22:17:31.546 INFO:teuthology.orchestra.run.smithi031.stdout:5253 writesame 0x4ae82fa thru 0x4aefc75 data_size 0x32(0x797c bytes) 2024-09-16T22:17:31.547 INFO:teuthology.orchestra.run.smithi031.stdout:5254 punch from 0xb53a30b to 0xb53a886, (0x57b bytes) 2024-09-16T22:17:31.548 INFO:teuthology.orchestra.run.smithi031.stdout:5255 write 0xbda8586 thru 0xbdac874 (0x42ef bytes) 2024-09-16T22:17:31.551 INFO:teuthology.orchestra.run.smithi031.stdout:5256 writesame 0x82ee00d thru 0x82f7995 data_size 0x7(0x9989 bytes) 2024-09-16T22:17:31.557 INFO:teuthology.orchestra.run.smithi031.stdout:5257 writesame 0xb374fce thru 0xb37b35e data_size 0x1(0x6391 bytes) 2024-09-16T22:17:31.558 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:31.558 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:31.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:31.576 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:31.578+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:31.583 INFO:teuthology.orchestra.run.smithi031.stdout:5258 trunc from 0xc472278 to 0x14834b7 2024-09-16T22:17:31.609 INFO:teuthology.orchestra.run.smithi031.stdout:5259 read 0x7068c7 thru 0x70e3c4 (0x7afe bytes) 2024-09-16T22:17:31.609 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.609 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.618 INFO:teuthology.orchestra.run.smithi031.stdout:5265 write 0x39bbb6c thru 0x39c357d (0x7a12 bytes) 2024-09-16T22:17:31.621 INFO:teuthology.orchestra.run.smithi031.stdout:5266 write 0x170dc35 thru 0x1716b1f (0x8eeb bytes) 2024-09-16T22:17:31.639 INFO:teuthology.orchestra.run.smithi031.stdout:5267 write 0xdcd0660 thru 0xdcdb5cd (0xaf6e bytes) 2024-09-16T22:17:31.644 INFO:teuthology.orchestra.run.smithi031.stdout:5268 write 0x34c47b3 thru 0x34ccf9d (0x87eb bytes) 2024-09-16T22:17:31.645 INFO:teuthology.orchestra.run.smithi031.stdout:5269 write 0x94a0e69 thru 0x94a3924 (0x2abc bytes) 2024-09-16T22:17:31.677 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:31.679+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:31.700 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.639520617999999, 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115259 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.618996878000001, 2024-09-16T22:17:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043168221, 2024-09-16T22:17:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115259 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082587+0000", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087294+0000", 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047076490000000004 2024-09-16T22:17:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087309+0000", 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4551e-05 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087318+0000", 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087372+0000", 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4184999999999997e-05 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087492+0000", 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119762 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087887+0000", 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039564899999999998 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.124754+0000", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036866492000000001 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125690+0000", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093677799999999998 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125726+0000", 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5160999999999998e-05 2024-09-16T22:17:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125752+0000", 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6177e-05 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131065 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 884736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981256297, 2024-09-16T22:17:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035774268999999997, 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131065 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720324+0000", 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-07 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720328+0000", 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0169999999999998e-06 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720332+0000", 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720334+0000", 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.542e-06 2024-09-16T22:17:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.755899+0000", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035564506000000003 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756010+0000", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111073 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756098+0000", 2024-09-16T22:17:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000001e-05 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131066 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 933888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981242879, 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035857077000000001, 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131066 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.46e-07 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720339+0000", 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0580000000000001e-06 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720343+0000", 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720346+0000", 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4360000000000001e-06 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756115+0000", 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035768988000000002 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756155+0000", 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0742999999999999e-05 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756195+0000", 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9270999999999997e-05 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131067 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 983040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981231719, 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036032743999999998, 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131067 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720350+0000", 2024-09-16T22:17:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720353+0000", 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720356+0000", 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5289999999999998e-06 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756226+0000", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035870446 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756303+0000", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6602999999999998e-05 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756382+0000", 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8951999999999999e-05 2024-09-16T22:17:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131068 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1032192~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981221849000001, 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036117121000000002, 2024-09-16T22:17:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131068 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720360+0000", 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720363+0000", 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720365+0000", 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4719999999999998e-06 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756397+0000", 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036031615000000003 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756437+0000", 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756476+0000", 2024-09-16T22:17:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8804999999999999e-05 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131101 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1196032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981212139, 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036281198000000001, 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131101 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720369+0000", 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720370+0000", 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.023e-06 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720373+0000", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720375+0000", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756508+0000", 2024-09-16T22:17:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036133086000000002 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756577+0000", 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9148999999999998e-05 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756650+0000", 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2290000000000001e-05 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131102 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1245184~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981200468000001, 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036372546999999998, 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131102 2024-09-16T22:17:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720382+0000", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7120000000000001e-06 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720388+0000", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720398+0000", 2024-09-16T22:17:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0433e-05 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756677+0000", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036278794000000003 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756716+0000", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9436e-05 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756753+0000", 2024-09-16T22:17:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6282000000000003e-05 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131118 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1294336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981176014000001, 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036544986000000002, 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131118 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720405+0000", 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720407+0000", 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0159999999999998e-06 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720413+0000", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6779999999999997e-06 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720417+0000", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4909999999999999e-06 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756785+0000", 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036368075 2024-09-16T22:17:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756867+0000", 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1853000000000004e-05 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756949+0000", 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2453000000000005e-05 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131171 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1056768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981056069999999, 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036546147000000001, 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131171 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720525+0000", 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-16T22:17:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720526+0000", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.029e-06 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720530+0000", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-06 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720533+0000", 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-16T22:17:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756986+0000", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036453754999999997 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757032+0000", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5531999999999997e-05 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757071+0000", 2024-09-16T22:17:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8751000000000002e-05 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131172 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1105920~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981044707000001, 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036728516000000003, 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131172 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720537+0000", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720540+0000", 2024-09-16T22:17:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720543+0000", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757109+0000", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036566047999999997 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757196+0000", 2024-09-16T22:17:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7019000000000002e-05 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757264+0000", 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8374000000000006e-05 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131173 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1155072~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981034559999999, 2024-09-16T22:17:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036824200000000001, 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131173 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-07 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720547+0000", 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0979999999999999e-06 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720550+0000", 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.129e-06 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720553+0000", 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.492e-06 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757280+0000", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036726795 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757329+0000", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9703000000000002e-05 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757370+0000", 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0748999999999999e-05 2024-09-16T22:17:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131174 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1204224~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.981000382, 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036974039, 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131174 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720582+0000", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.531e-06 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720586+0000", 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:17:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720589+0000", 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8040000000000002e-06 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757402+0000", 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036813564999999999 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757480+0000", 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8103999999999996e-05 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757554+0000", 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3825999999999994e-05 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131175 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1253376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980987671999999, 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037074818000000002, 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131175 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3000000000000002e-07 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720594+0000", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720597+0000", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720600+0000", 2024-09-16T22:17:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757586+0000", 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036985393999999998 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757628+0000", 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2644e-05 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757668+0000", 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9307000000000002e-05 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131176 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1302528~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980977098, 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037249833000000003, 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131176 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720604+0000", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720605+0000", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7199999999999997e-07 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720608+0000", 2024-09-16T22:17:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.869e-06 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720610+0000", 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757701+0000", 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037091164000000003 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757780+0000", 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.852e-05 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757853+0000", 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3368999999999995e-05 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131177 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1089536~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980967219, 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037365571, 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131177 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720615+0000", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.034e-06 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720617+0000", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720620+0000", 2024-09-16T22:17:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757885+0000", 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037265057999999997 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757935+0000", 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0112999999999999e-05 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757979+0000", 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3584000000000001e-05 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131178 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1138688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980957321, 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037553098, 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131178 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3099999999999999e-07 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720630+0000", 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-16T22:17:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720633+0000", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2160000000000002e-06 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720636+0000", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758017+0000", 2024-09-16T22:17:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037380913000000002 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758097+0000", 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0029000000000002e-05 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758176+0000", 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9716999999999996e-05 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131179 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1187840~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980941387, 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037658184999999997, 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131179 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720640+0000", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.037e-06 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720644+0000", 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0469999999999998e-06 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720646+0000", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5009999999999999e-06 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758204+0000", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037558084999999998 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758254+0000", 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9777000000000001e-05 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758297+0000", 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3405000000000002e-05 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131180 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1236992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980931451, 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037835806, 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131180 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720650+0000", 2024-09-16T22:17:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031e-06 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720653+0000", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.892e-06 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720656+0000", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.339e-06 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758335+0000", 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037679556000000003 2024-09-16T22:17:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758409+0000", 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3980000000000007e-05 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758485+0000", 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5797999999999995e-05 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131181 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1286144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.980921951999999, 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037949063999999998, 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131181 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4299999999999999e-07 2024-09-16T22:17:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720660+0000", 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720663+0000", 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720665+0000", 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758515+0000", 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037850097999999999 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758564+0000", 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8179000000000001e-05 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758608+0000", 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4088000000000002e-05 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.801 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:31.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5270 read 0xc3406fb thru 0xc34ea9d (0xe3a3 bytes) 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5271 write 0x8b2f219 thru 0x8b3a635 (0xb41d bytes) 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5274 punch from 0x6d38290 to 0x6d390f3, (0xe63 bytes) 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5277 read 0x8032123 thru 0x803e187 (0xc065 bytes) 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5278 trunc from 0xdcdb5ce to 0xe13db3e 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5279 writesame 0x8c596da thru 0x8c622ba data_size 0x1(0x8be1 bytes) 2024-09-16T22:17:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:5280 trunc from 0xe13db3e to 0xa1f51f9 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5281 write 0x6935c20 thru 0x693a0f3 (0x44d4 bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5283 trunc from 0xa1f51f9 to 0xe5c80d4 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5286 punch from 0xb0d3c05 to 0xb0d56ea, (0x1ae5 bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5290 trunc from 0xe5c80d4 to 0xb96db35 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5291 write 0x1ef5c34 thru 0x1f0082e (0xabfb bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5292 write 0x3ac7c1c thru 0x3ad16a5 (0x9a8a bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5295 write 0xc38ceed thru 0xc39217e (0x5292 bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5296 punch from 0x484e805 to 0x4850d4c, (0x2547 bytes) 2024-09-16T22:17:31.806 INFO:teuthology.orchestra.run.smithi031.stdout:5297 write 0xca37788 thru 0xca44689 (0xcf02 bytes) 2024-09-16T22:17:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:5298 punch from 0x260b7bf to 0x260ea2a, (0x326b bytes) 2024-09-16T22:17:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:5299 read 0xb46e74c thru 0xb479273 (0xab28 bytes) 2024-09-16T22:17:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:5300 read 0x8d5ea4c thru 0x8d5f474 (0xa29 bytes) 2024-09-16T22:17:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:5301 read 0x9ddaef1 thru 0x9de12dd (0x63ed bytes) 2024-09-16T22:17:31.807 INFO:teuthology.orchestra.run.smithi031.stdout:5302 write 0xf27324 thru 0xf29f8c (0x2c69 bytes) 2024-09-16T22:17:31.807 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:31.804+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:31.808 INFO:teuthology.orchestra.run.smithi031.stdout:5303 read 0x2e86225 thru 0x2e9534d (0xf129 bytes) 2024-09-16T22:17:31.808 INFO:teuthology.orchestra.run.smithi031.stdout:5305 read 0x27665fe thru 0x276c390 (0x5d93 bytes) 2024-09-16T22:17:31.808 INFO:teuthology.orchestra.run.smithi031.stdout:5306 trunc from 0xca4468a to 0x9829a8d 2024-09-16T22:17:31.819 INFO:teuthology.orchestra.run.smithi031.stdout:5307 trunc from 0x9829a8d to 0x9567a55 2024-09-16T22:17:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:5308 read 0x5e18722 thru 0x5e2402c (0xb90b bytes) 2024-09-16T22:17:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:5309 write 0xc981b6d thru 0xc9883c2 (0x6856 bytes) 2024-09-16T22:17:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:5310 trunc from 0xc9883c3 to 0x54a40a2 2024-09-16T22:17:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:5311 read 0x25767b7 thru 0x257fcfe (0x9548 bytes) 2024-09-16T22:17:31.864 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:31.865 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:31.865 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T22:17:31.865 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:31.865 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:31.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T22:17:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:5312 writesame 0x6f71e1a thru 0x6f7ce47 data_size 0x3(0xb02e bytes) 2024-09-16T22:17:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:5313 writesame 0x8ad5937 thru 0x8ad7eff data_size 0x1(0x25c9 bytes) 2024-09-16T22:17:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:5314 writesame 0x2edcdf0 thru 0x2ee190d data_size 0xf(0x4b1e bytes) 2024-09-16T22:17:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:5315 punch from 0x53107fc to 0x531da9f, (0xd2a3 bytes) 2024-09-16T22:17:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:5316 trunc from 0x8ad7f00 to 0xb5065cc 2024-09-16T22:17:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:5317 trunc from 0xb5065cc to 0x158de23 2024-09-16T22:17:31.903 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:31.905+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:31.926 INFO:teuthology.orchestra.run.smithi031.stdout:5318 write 0x1f85a66 thru 0x1f92302 (0xc89d bytes) 2024-09-16T22:17:31.933 INFO:teuthology.orchestra.run.smithi031.stdout:5319 write 0x219df15 thru 0x21aa32f (0xc41b bytes) 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185577 3.f e84/23 3:f70f30da:::rbd_object_map.11cded7d7aa9:head v 84'568, mlcod=84'566)", 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.936630+0000", 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00070851400000000004, 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072340600000000001, 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936630+0000", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936630+0000", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936631+0000", 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1999999999999999e-06 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936638+0000", 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5699999999999998e-06 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936650+0000", 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2301e-05 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936657+0000", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.883e-06 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936711+0000", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4669999999999997e-05 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.936732+0000", 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0678000000000001e-05 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:31.939 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:5321 writesame 0x8d57cca thru 0x8d5fbb5 data_size 0x2(0x7eec bytes) 2024-09-16T22:17:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:5322 punch from 0x6a08879 to 0x6a182ff, (0xfa86 bytes) 2024-09-16T22:17:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:5323 writesame 0x28f824b thru 0x2906374 data_size 0x27(0xe12a bytes) 2024-09-16T22:17:31.953 INFO:teuthology.orchestra.run.smithi031.stdout:5324 writesame 0x234e7c0 thru 0x2354411 data_size 0x4e(0x5c52 bytes) 2024-09-16T22:17:31.960 INFO:teuthology.orchestra.run.smithi031.stdout:5325 write 0x94076fd thru 0x940f8e3 (0x81e7 bytes) 2024-09-16T22:17:31.964 INFO:teuthology.orchestra.run.smithi031.stdout:5326 read 0x8904653 thru 0x8911f6d (0xd91b bytes) 2024-09-16T22:17:31.964 INFO:teuthology.orchestra.run.smithi031.stdout:5329 punch from 0x4aad771 to 0x4abba76, (0xe305 bytes) 2024-09-16T22:17:31.964 INFO:teuthology.orchestra.run.smithi031.stdout:5330 punch from 0x3b8c5af to 0x3b9ab84, (0xe5d5 bytes) 2024-09-16T22:17:31.964 INFO:teuthology.orchestra.run.smithi031.stdout:5334 write 0x88ce8b7 thru 0x88d04c8 (0x1c12 bytes) 2024-09-16T22:17:31.967 INFO:teuthology.orchestra.run.smithi031.stdout:5336 write 0x6f09743 thru 0x6f11b50 (0x840e bytes) 2024-09-16T22:17:31.979 INFO:teuthology.orchestra.run.smithi031.stdout:5337 writesame 0xbcb7e3b thru 0xbcc2f82 data_size 0x3e(0xb148 bytes) 2024-09-16T22:17:31.979 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:31.983 INFO:teuthology.orchestra.run.smithi031.stdout:5339 trunc from 0xbcc2f83 to 0x88aed14 2024-09-16T22:17:31.999 INFO:teuthology.orchestra.run.smithi031.stdout:5342 write 0xacdce05 thru 0xace1248 (0x4444 bytes) 2024-09-16T22:17:31.999 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:17:32.000 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.003 INFO:teuthology.orchestra.run.smithi031.stdout:5344 read 0x3e6528 thru 0x3f2927 (0xc400 bytes) 2024-09-16T22:17:32.003 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:32.005+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:32.003 INFO:teuthology.orchestra.run.smithi031.stdout:5345 write 0x7cc3c8b thru 0x7cc7f62 (0x42d8 bytes) 2024-09-16T22:17:32.005 INFO:teuthology.orchestra.run.smithi031.stdout:5346 read 0x9be9ed0 thru 0x9bf244f (0x8580 bytes) 2024-09-16T22:17:32.006 INFO:teuthology.orchestra.run.smithi031.stdout:5347 punch from 0x658328e to 0x6588143, (0x4eb5 bytes) 2024-09-16T22:17:32.006 INFO:teuthology.orchestra.run.smithi031.stdout:5348 read 0x651cfad thru 0x6522a1e (0x5a72 bytes) 2024-09-16T22:17:32.006 INFO:teuthology.orchestra.run.smithi031.stdout:5349 read 0x3b54af4 thru 0x3b58c47 (0x4154 bytes) 2024-09-16T22:17:32.015 INFO:teuthology.orchestra.run.smithi031.stdout:5351 write 0xd4ff847 thru 0xd509970 (0xa12a bytes) 2024-09-16T22:17:32.019 INFO:teuthology.orchestra.run.smithi031.stdout:5352 read 0x2bf1947 thru 0x2bfb4dc (0x9b96 bytes) 2024-09-16T22:17:32.019 INFO:teuthology.orchestra.run.smithi031.stdout:5353 read 0x6cc584a thru 0x6cc7ef8 (0x26af bytes) 2024-09-16T22:17:32.019 INFO:teuthology.orchestra.run.smithi031.stdout:5354 writesame 0xa6fcb41 thru 0xa701f1b data_size 0x1(0x53db bytes) 2024-09-16T22:17:32.030 INFO:teuthology.orchestra.run.smithi031.stdout:5356 punch from 0x9d661f5 to 0x9d702e9, (0xa0f4 bytes) 2024-09-16T22:17:32.030 INFO:teuthology.orchestra.run.smithi031.stdout:5357 writesame 0x70b17f thru 0x717591 data_size 0x1(0xc413 bytes) 2024-09-16T22:17:32.067 INFO:teuthology.orchestra.run.smithi031.stdout:5358 trunc from 0xd509971 to 0x393bb9e 2024-09-16T22:17:32.071 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:32.071 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:32.096 INFO:teuthology.orchestra.run.smithi031.stdout:5361 punch from 0x172e01e to 0x1739093, (0xb075 bytes) 2024-09-16T22:17:32.096 INFO:teuthology.orchestra.run.smithi031.stdout:5362 write 0x20a6157 thru 0x20addf9 (0x7ca3 bytes) 2024-09-16T22:17:32.100 INFO:teuthology.orchestra.run.smithi031.stdout:5363 trunc from 0x393bb9e to 0x6447633 2024-09-16T22:17:32.104 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:32.106+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:32.109 INFO:teuthology.orchestra.run.smithi031.stdout:5365 trunc from 0x6447633 to 0x405219f 2024-09-16T22:17:32.116 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:32.132 INFO:teuthology.orchestra.run.smithi031.stdout:5367 write 0xc857df9 thru 0xc85e50f (0x6717 bytes) 2024-09-16T22:17:32.135 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:32.135 INFO:teuthology.orchestra.run.smithi031.stdout:5374 read 0x6537c39 thru 0x6547722 (0xfaea bytes) 2024-09-16T22:17:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:5375 trunc from 0xc85e510 to 0x3f2d14c 2024-09-16T22:17:32.158 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:32.171 INFO:teuthology.orchestra.run.smithi031.stdout:5379 writesame 0x9e69a72 thru 0x9e69e71 data_size 0x10(0x400 bytes) 2024-09-16T22:17:32.171 INFO:teuthology.orchestra.run.smithi031.stdout:5380 read 0x5823e95 thru 0x582a76c (0x68d8 bytes) 2024-09-16T22:17:32.171 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:32.178 INFO:teuthology.orchestra.run.smithi031.stdout:5382 write 0xb0c36b5 thru 0xb0c7d00 (0x464c bytes) 2024-09-16T22:17:32.183 INFO:teuthology.orchestra.run.smithi031.stdout:5384 trunc from 0xb0c7d01 to 0xcaece5b 2024-09-16T22:17:32.189 INFO:teuthology.orchestra.run.smithi031.stdout:5388 write 0xdb24d9 thru 0xdb6351 (0x3e79 bytes) 2024-09-16T22:17:32.198 INFO:teuthology.orchestra.run.smithi031.stdout:5390 trunc from 0xcaece5b to 0x54aba68 2024-09-16T22:17:32.205 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:32.206+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:32.226 INFO:teuthology.orchestra.run.smithi031.stdout:5391 write 0xda29d71 thru 0xda35b66 (0xbdf6 bytes) 2024-09-16T22:17:32.230 INFO:teuthology.orchestra.run.smithi031.stdout:5392 write 0x3c91e4e thru 0x3c9234c (0x4ff bytes) 2024-09-16T22:17:32.231 INFO:teuthology.orchestra.run.smithi031.stdout:5394 writesame 0x37e15d4 thru 0x37e8ff0 data_size 0x1(0x7a1d bytes) 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55222 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4317184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.897393112000003, 2024-09-16T22:17:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063938485000000003, 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55222 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336551+0000", 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336759+0000", 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000208063 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336764+0000", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8720000000000001e-06 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336776+0000", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.182e-05 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336782+0000", 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7789999999999998e-06 2024-09-16T22:17:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400086+0000", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063303645000000006 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400310+0000", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022399299999999999 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400489+0000", 2024-09-16T22:17:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017931300000000001 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55223 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.897149779000003, 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911932000000005, 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55223 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336794+0000", 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336950+0000", 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155821 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336954+0000", 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2490000000000003e-06 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336965+0000", 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1055e-05 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336972+0000", 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4309999999999999e-06 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400531+0000", 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559177999999994 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400612+0000", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1360999999999996e-05 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400706+0000", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3837000000000003e-05 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55224 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4333568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.896953672000002, 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064193369, 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55224 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.336990+0000", 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337203+0000", 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021318899999999999 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337208+0000", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1710000000000004e-06 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337215+0000", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3089999999999998e-06 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337219+0000", 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-16T22:17:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.400749+0000", 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063530160000000002 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401000+0000", 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000250995 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401184+0000", 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018346399999999999 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55225 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4919296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.896716861000002, 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064167392000000004, 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55225 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337227+0000", 2024-09-16T22:17:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337451+0000", 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022388599999999999 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337456+0000", 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7280000000000003e-06 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337466+0000", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0351e-05 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337473+0000", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7449999999999998e-06 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401218+0000", 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063745183999999996 2024-09-16T22:17:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401298+0000", 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0302999999999993e-05 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401394+0000", 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6194999999999996e-05 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55226 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4349952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.896459704999998, 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064121972999999999, 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55226 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337484+0000", 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337709+0000", 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022441200000000001 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337713+0000", 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337724+0000", 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0740999999999999e-05 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337731+0000", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401426+0000", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063695389000000005 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401507+0000", 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0356000000000001e-05 2024-09-16T22:17:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401606+0000", 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9631999999999997e-05 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55227 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4935680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.896200964999998, 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064076279, 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55227 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337743+0000", 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337889+0000", 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000146055 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337894+0000", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779999999999999e-06 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337901+0000", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4669999999999998e-06 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337905+0000", 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1080000000000003e-06 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401637+0000", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063731333000000001 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401721+0000", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3969999999999997e-05 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401819+0000", 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8667999999999998e-05 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55228 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4366336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.896030512999999, 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064169421000000004, 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55228 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.337913+0000", 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338139+0000", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022557499999999999 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338143+0000", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338158+0000", 2024-09-16T22:17:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4498000000000001e-05 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338165+0000", 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401880+0000", 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063715331 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.401978+0000", 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8158000000000004e-05 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402083+0000", 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104799 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55229 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.895691587999998, 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064053115999999993, 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55229 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338252+0000", 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338426+0000", 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017367500000000001 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338431+0000", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338442+0000", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1749999999999999e-05 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338449+0000", 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6649999999999997e-06 2024-09-16T22:17:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402117+0000", 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063668279999999994 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402207+0000", 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9257000000000003e-05 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402305+0000", 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8911000000000002e-05 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55231 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4968448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.895113397999999, 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063911281, 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55231 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338831+0000", 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-07 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338834+0000", 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338841+0000", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338847+0000", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1360000000000002e-06 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402566+0000", 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063718876999999993 2024-09-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402647+0000", 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1384000000000005e-05 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402742+0000", 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4686999999999997e-05 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55232 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4399104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.895088395999998, 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064122381000000006, 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55232 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338856+0000", 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339005+0000", 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014984300000000001 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339010+0000", 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999999e-06 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339023+0000", 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2904e-05 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339030+0000", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2660000000000004e-06 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402777+0000", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063746612999999994 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402864+0000", 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7818000000000005e-05 2024-09-16T22:17:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.402978+0000", 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113507 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55233 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4984832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.894902450000004, 2024-09-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151505999999997, 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55233 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339041+0000", 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339210+0000", 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016822699999999999 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.741e-06 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339225+0000", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0149000000000001e-05 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339232+0000", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403011+0000", 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063779340000000004 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403096+0000", 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5482000000000006e-05 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403193+0000", 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6496999999999994e-05 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55234 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4415488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.894696087, 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064151969000000003, 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55234 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339248+0000", 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339412+0000", 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000164544 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339417+0000", 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6160000000000004e-06 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339428+0000", 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0522e-05 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339435+0000", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7880000000000007e-06 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403227+0000", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063791688999999999 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403309+0000", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2254000000000002e-05 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403400+0000", 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0556000000000005e-05 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55235 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5001216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.894496494000002, 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064168931999999998, 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55235 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339447+0000", 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339684+0000", 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023680199999999999 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339690+0000", 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.049e-06 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339701+0000", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0494e-05 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339706+0000", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6579999999999999e-06 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403431+0000", 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063724905999999998 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403519+0000", 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7892999999999993e-05 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403616+0000", 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.713e-05 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55236 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4431872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.894226838000002, 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064115839999999993, 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55236 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339717+0000", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339965+0000", 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024771199999999998 2024-09-16T22:17:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339970+0000", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339980+0000", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0254999999999999e-05 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339987+0000", 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7519999999999999e-06 2024-09-16T22:17:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403650+0000", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063663553999999997 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403736+0000", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5778000000000004e-05 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403833+0000", 2024-09-16T22:17:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6854000000000005e-05 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55237 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5017600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.893944877000003, 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064058452000000002, 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55237 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339999+0000", 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340000+0000", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340002+0000", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5859999999999999e-06 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340009+0000", 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2080000000000003e-06 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340015+0000", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403865+0000", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063850812000000007 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.403955+0000", 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9648999999999995e-05 2024-09-16T22:17:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404057+0000", 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102472 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55238 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4448256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.893920299000001, 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064252408999999996, 2024-09-16T22:17:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55238 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340024+0000", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340290+0000", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026611899999999999 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340294+0000", 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2810000000000002e-06 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340302+0000", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4630000000000001e-06 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340308+0000", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8259999999999998e-06 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404092+0000", 2024-09-16T22:17:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063783230999999996 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404178+0000", 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6080000000000003e-05 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404276+0000", 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8408999999999999e-05 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55239 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5033984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.893623748000003, 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064172565000000001, 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55239 2024-09-16T22:17:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.340320+0000", 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341020+0000", 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070000599999999998 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341025+0000", 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6589999999999998e-06 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341038+0000", 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3534e-05 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341046+0000", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6179999999999997e-06 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404308+0000", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063261957999999993 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404393+0000", 2024-09-16T22:17:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4791999999999997e-05 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.404493+0000", 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9998e-05 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55243 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5066752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.892760283999998, 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064032600999999995, 2024-09-16T22:17:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55243 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341184+0000", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341467+0000", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028289899999999998 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341471+0000", 2024-09-16T22:17:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5569999999999997e-06 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341481+0000", 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0339e-05 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341488+0000", 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405047+0000", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063559063999999998 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405129+0000", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1919e-05 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405216+0000", 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7230000000000003e-05 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55244 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 4497408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.892444597000001, 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063903336000000005, 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55244 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341499+0000", 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341673+0000", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017377700000000001 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341678+0000", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341688+0000", 2024-09-16T22:17:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0200000000000001e-05 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341695+0000", 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.55e-06 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405249+0000", 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063554352999999994 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405323+0000", 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3677000000000006e-05 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405403+0000", 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9628000000000004e-05 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:55245 3.1a 3:5b1cbae4:::rbd_data.11cd13540b0c.0000000000000001:1a [read 5083136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.892235493000001, 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063879072999999995, 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55245 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341708+0000", 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341930+0000", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022177199999999999 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341940+0000", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341950+0000", 2024-09-16T22:17:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0124e-05 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.341959+0000", 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7260000000000008e-06 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405436+0000", 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063477512 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405509+0000", 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2156999999999994e-05 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.405587+0000", 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8957999999999999e-05 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.324 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:32.328 INFO:teuthology.orchestra.run.smithi031.stdout:5395 write 0x5bd0ab4 thru 0x5bd1e33 (0x1380 bytes) 2024-09-16T22:17:32.328 INFO:teuthology.orchestra.run.smithi031.stdout:5396 writesame 0x24a96c1 thru 0x24b8da4 data_size 0x8a(0xf6e4 bytes) 2024-09-16T22:17:32.328 INFO:teuthology.orchestra.run.smithi031.stdout:5397 write 0x7214eaf thru 0x7222add (0xdc2f bytes) 2024-09-16T22:17:32.328 INFO:teuthology.orchestra.run.smithi031.stdout:5398 write 0x53772d thru 0x540f77 (0x984b bytes) 2024-09-16T22:17:32.328 INFO:teuthology.orchestra.run.smithi031.stdout:5399 read 0x3de4af4 thru 0x3de69b8 (0x1ec5 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5400 write 0x362e2a6 thru 0x3638d08 (0xaa63 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5401 write 0x733e9fd thru 0x734c1a7 (0xd7ab bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5402 write 0x1f2cf3f thru 0x1f35c95 (0x8d57 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5403 punch from 0xa1c04a9 to 0xa1c4525, (0x407c bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5404 writesame 0x85a3fc8 thru 0x85b15bc data_size 0x1(0xd5f5 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5406 writesame 0x5218e9c thru 0x52213d9 data_size 0x2d(0x853e bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5407 writesame 0x1ded9e thru 0x1e778d data_size 0x8(0x89f0 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5409 punch from 0xc1d40bf to 0xc1de39d, (0xa2de bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5411 writesame 0x939b0f0 thru 0x939b6e9 data_size 0x12(0x5fa bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5412 read 0x9451a5a thru 0x9457370 (0x5917 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5413 write 0xd17982d thru 0xd188b90 (0xf364 bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5414 writesame 0x7284205 thru 0x728b312 data_size 0x1d(0x710e bytes) 2024-09-16T22:17:32.329 INFO:teuthology.orchestra.run.smithi031.stdout:5415 trunc from 0xda35b67 to 0x2c54b22 2024-09-16T22:17:32.330 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:17:32.330 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:32.327+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:32.336 INFO:teuthology.orchestra.run.smithi031.stdout:5416 read 0xfe1fd1 thru 0xfe6996 (0x49c6 bytes) 2024-09-16T22:17:32.336 INFO:teuthology.orchestra.run.smithi031.stdout:5417 clone 43 order 22 su 131072 sc 14 2024-09-16T22:17:32.427 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:32.428+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:32.442 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:32.443 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:32.528 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:32.530+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:32.558 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:32.559 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:32.629 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:32.631+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.099021991000001, 2024-09-16T22:17:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.099002650999999, 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.098598250999999, 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45048 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.096924805999997, 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676787000000002, 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45048 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579107+0000", 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579108+0000", 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579110+0000", 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579116+0000", 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0229999999999998e-06 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579120+0000", 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630608+0000", 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051487796000000002 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630687+0000", 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8709999999999997e-05 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630784+0000", 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7190999999999998e-05 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.096902980999999, 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.096872447999999, 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.092471549000003, 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.092451068000003, 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.092430491999998, 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.092406007000001, 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.092384436000003, 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086305838000001, 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086285506000003, 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086264286999999, 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086242915, 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086222077000002, 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086201496000001, 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.086179928999996, 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.081533055000001, 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0262276539999999, 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:17:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:17:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:32.767 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:32.769 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:32.770+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:32.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:32.869 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:32.870+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:32.924 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:32.924 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:32.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:32.969 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:32.971+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:33.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:33.022 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:33.022 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:33.038 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:33.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:33.052 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:33.052 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:33.055 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:33.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:33.071 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:33.072+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:33.072 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:33.072 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:33.090 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:33.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:33.107 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:33.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:33.125 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:33.169 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.170 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.172 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:33.173+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:33.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:33.272 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:33.274+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:33.294 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.294 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:33.373 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:33.375+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-16T22:17:33.473 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:33.475+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:33.537 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.537 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:33.598 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:33.588+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:33.675 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:33.677+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:33.720 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:33.738 INFO:teuthology.orchestra.run.smithi031.stdout:5418 write 0x5155699 thru 0x515a647 (0x4faf bytes) 2024-09-16T22:17:33.745 INFO:teuthology.orchestra.run.smithi031.stdout:5420 trunc from 0x515a648 to 0x1f80e25 2024-09-16T22:17:33.776 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:33.777+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:33.804 INFO:teuthology.orchestra.run.smithi031.stdout:5421 read 0x159acf3 thru 0x15a6b37 (0xbe45 bytes) 2024-09-16T22:17:33.824 INFO:teuthology.orchestra.run.smithi031.stdout:5422 write 0xe59e845 thru 0xe5a5706 (0x6ec2 bytes) 2024-09-16T22:17:33.828 INFO:teuthology.orchestra.run.smithi031.stdout:5423 write 0x736d743 thru 0x73760cd (0x898b bytes) 2024-09-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi031.stdout:5424 write 0x72b122b thru 0x72b5e9c (0x4c72 bytes) 2024-09-16T22:17:33.835 INFO:teuthology.orchestra.run.smithi031.stdout:5425 trunc from 0xe5a5707 to 0xcdd4b00 2024-09-16T22:17:33.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:5430 write 0x4957abb thru 0x4962b33 (0xb079 bytes) 2024-09-16T22:17:33.849 INFO:teuthology.orchestra.run.smithi031.stdout:5431 punch from 0xaeb7b77 to 0xaec393a, (0xbdc3 bytes) 2024-09-16T22:17:33.849 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.856 INFO:teuthology.orchestra.run.smithi031.stdout:5433 writesame 0xdea1b81 thru 0xdeab18d data_size 0x1(0x960d bytes) 2024-09-16T22:17:33.856 INFO:teuthology.orchestra.run.smithi031.stdout:5434 writesame 0x1732c8e thru 0x1733de5 data_size 0x28(0x1158 bytes) 2024-09-16T22:17:33.857 INFO:teuthology.orchestra.run.smithi031.stdout:5435 write 0x67ad3d6 thru 0x67b6d2c (0x9957 bytes) 2024-09-16T22:17:33.860 INFO:teuthology.orchestra.run.smithi031.stdout:5438 writesame 0x3f66e09 thru 0x3f70deb data_size 0xb(0x9fe3 bytes) 2024-09-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi031.stdout:5439 write 0xd72a3bc thru 0xd731ea2 (0x7ae7 bytes) 2024-09-16T22:17:33.866 INFO:teuthology.orchestra.run.smithi031.stdout:5440 read 0x869b888 thru 0x86a7dd1 (0xc54a bytes) 2024-09-16T22:17:33.866 INFO:teuthology.orchestra.run.smithi031.stdout:5442 read 0x3b5965d thru 0x3b5a4d3 (0xe77 bytes) 2024-09-16T22:17:33.876 INFO:teuthology.orchestra.run.smithi031.stdout:5444 trunc from 0xdeab18e to 0x52797df 2024-09-16T22:17:33.877 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:33.878+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:33.877 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.877 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:33.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:33.890 INFO:teuthology.orchestra.run.smithi031.stdout:5445 punch from 0x14ac269 to 0x14b0571, (0x4308 bytes) 2024-09-16T22:17:33.890 INFO:teuthology.orchestra.run.smithi031.stdout:5446 punch from 0x17e3fdc to 0x17f251e, (0xe542 bytes) 2024-09-16T22:17:33.891 INFO:teuthology.orchestra.run.smithi031.stdout:5447 read 0x3335724 thru 0x33357c6 (0xa3 bytes) 2024-09-16T22:17:33.893 INFO:teuthology.orchestra.run.smithi031.stdout:5448 trunc from 0x52797df to 0x24eb554 2024-09-16T22:17:33.906 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.907 INFO:teuthology.orchestra.run.smithi031.stdout:5450 read 0x1f11bc9 thru 0x1f17e5c (0x6294 bytes) 2024-09-16T22:17:33.915 INFO:teuthology.orchestra.run.smithi031.stdout:5451 write 0x5418b4b thru 0x542290e (0x9dc4 bytes) 2024-09-16T22:17:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:5453 trunc from 0x542290f to 0x3a2e78d 2024-09-16T22:17:33.935 INFO:teuthology.orchestra.run.smithi031.stdout:5454 writesame 0x4d49a9f thru 0x4d559c1 data_size 0x1(0xbf23 bytes) 2024-09-16T22:17:33.935 INFO:teuthology.orchestra.run.smithi031.stdout:5455 read 0x23dc01c thru 0x23e4cfa (0x8cdf bytes) 2024-09-16T22:17:33.943 INFO:teuthology.orchestra.run.smithi031.stdout:5456 trunc from 0x4d559c2 to 0xc1620f9 2024-09-16T22:17:33.950 INFO:teuthology.orchestra.run.smithi031.stdout:5457 trunc from 0xc1620f9 to 0xcfe021e 2024-09-16T22:17:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:5458 punch from 0x118cbc7 to 0x1193405, (0x683e bytes) 2024-09-16T22:17:33.957 INFO:teuthology.orchestra.run.smithi031.stdout:5459 trunc from 0xcfe021e to 0xb12bfb4 2024-09-16T22:17:33.963 INFO:teuthology.orchestra.run.smithi031.stdout:5460 writesame 0x9127e35 thru 0x9134189 data_size 0x49(0xc355 bytes) 2024-09-16T22:17:33.966 INFO:teuthology.orchestra.run.smithi031.stdout:5461 write 0xe835ab thru 0xe9277a (0xf1d0 bytes) 2024-09-16T22:17:33.967 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.967 INFO:teuthology.orchestra.run.smithi031.stdout:5466 punch from 0x70b2d1b to 0x70c1d3a, (0xf01f bytes) 2024-09-16T22:17:33.970 INFO:teuthology.orchestra.run.smithi031.stdout:5468 trunc from 0xb12bfb4 to 0xdd79b2f 2024-09-16T22:17:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:5469 read 0x4385265 thru 0x438a62f (0x53cb bytes) 2024-09-16T22:17:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:5471 read 0x90ebea2 thru 0x90f7b2a (0xbc89 bytes) 2024-09-16T22:17:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:5472 punch from 0x7fd04a8 to 0x7fd80d5, (0x7c2d bytes) 2024-09-16T22:17:33.976 INFO:teuthology.orchestra.run.smithi031.stdout:5473 writesame 0x229e89d thru 0x22adf73 data_size 0x1(0xf6d7 bytes) 2024-09-16T22:17:33.977 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:33.978+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:33.991 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:33.991 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:34.005 INFO:teuthology.orchestra.run.smithi031.stdout:5474 writesame 0xdacc901 thru 0xdad9181 data_size 0x1(0xc881 bytes) 2024-09-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:5475 punch from 0xa7d3b6e to 0xa7d6e81, (0x3313 bytes) 2024-09-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:5479 read 0xb01138c thru 0xb020446 (0xf0bb bytes) 2024-09-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:5481 write 0x98d7a4d thru 0x98dcf8e (0x5542 bytes) 2024-09-16T22:17:34.037 INFO:teuthology.orchestra.run.smithi031.stdout:5482 write 0x15ab5 thru 0x20717 (0xac63 bytes) 2024-09-16T22:17:34.039 INFO:teuthology.orchestra.run.smithi031.stdout:5483 trunc from 0xdd79b2f to 0xdcf5df0 2024-09-16T22:17:34.051 INFO:teuthology.orchestra.run.smithi031.stdout:5484 trunc from 0xdcf5df0 to 0x6dfc41b 2024-09-16T22:17:34.075 INFO:teuthology.orchestra.run.smithi031.stdout:5485 writesame 0xdee2d9c thru 0xdef20df data_size 0x2(0xf344 bytes) 2024-09-16T22:17:34.075 INFO:teuthology.orchestra.run.smithi031.stdout:5486 write 0xd3bde97 thru 0xd3caf01 (0xd06b bytes) 2024-09-16T22:17:34.078 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:34.080+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:34.079 INFO:teuthology.orchestra.run.smithi031.stdout:5487 write 0xcaecd96 thru 0xcaf2500 (0x576b bytes) 2024-09-16T22:17:34.081 INFO:teuthology.orchestra.run.smithi031.stdout:5488 read 0xbd6b80b thru 0xbd7b5b9 (0xfdaf bytes) 2024-09-16T22:17:34.081 INFO:teuthology.orchestra.run.smithi031.stdout:5489 write 0xd8c7857 thru 0xd8c8c0c (0x13b6 bytes) 2024-09-16T22:17:34.084 INFO:teuthology.orchestra.run.smithi031.stdout:5490 read 0x83aabc4 thru 0x83b0729 (0x5b66 bytes) 2024-09-16T22:17:34.084 INFO:teuthology.orchestra.run.smithi031.stdout:5491 read 0xc2db55 thru 0xc3c364 (0xe810 bytes) 2024-09-16T22:17:34.085 INFO:teuthology.orchestra.run.smithi031.stdout:5493 writesame 0x5203242 thru 0x5207c21 data_size 0x30(0x49e0 bytes) 2024-09-16T22:17:34.088 INFO:teuthology.orchestra.run.smithi031.stdout:5494 writesame 0xb7d5d0c thru 0xb7db05a data_size 0x1(0x534f bytes) 2024-09-16T22:17:34.100 INFO:teuthology.orchestra.run.smithi031.stdout:5496 writesame 0x8c3844e thru 0x8c3a6d0 data_size 0x39(0x2283 bytes) 2024-09-16T22:17:34.105 INFO:teuthology.orchestra.run.smithi031.stdout:5497 read 0x247eca4 thru 0x248b0fe (0xc45b bytes) 2024-09-16T22:17:34.105 INFO:teuthology.orchestra.run.smithi031.stdout:5498 write 0x2616fad thru 0x261c266 (0x52ba bytes) 2024-09-16T22:17:34.105 INFO:teuthology.orchestra.run.smithi031.stdout:5499 write 0x5b52446 thru 0x5b559fc (0x35b7 bytes) 2024-09-16T22:17:34.107 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.108 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:34.119 INFO:teuthology.orchestra.run.smithi031.stdout:5500 trunc from 0xdef20e0 to 0x423701d 2024-09-16T22:17:34.130 INFO:teuthology.orchestra.run.smithi031.stdout:5504 writesame 0x2f177dc thru 0x2f26c6c data_size 0x1(0xf491 bytes) 2024-09-16T22:17:34.154 INFO:teuthology.orchestra.run.smithi031.stdout:5505 write 0x5a6e2eb thru 0x5a7c4fd (0xe213 bytes) 2024-09-16T22:17:34.157 INFO:teuthology.orchestra.run.smithi031.stdout:5506 punch from 0x4833dce to 0x48399d0, (0x5c02 bytes) 2024-09-16T22:17:34.157 INFO:teuthology.orchestra.run.smithi031.stdout:5507 read 0x3a49a3d thru 0x3a58dac (0xf370 bytes) 2024-09-16T22:17:34.162 INFO:teuthology.orchestra.run.smithi031.stdout:5508 trunc from 0x5a7c4fe to 0x9977f2c 2024-09-16T22:17:34.168 INFO:teuthology.orchestra.run.smithi031.stdout:5510 punch from 0x1491a55 to 0x14a0844, (0xedef bytes) 2024-09-16T22:17:34.168 INFO:teuthology.orchestra.run.smithi031.stdout:5513 punch from 0x1d30c22 to 0x1d32d96, (0x2174 bytes) 2024-09-16T22:17:34.168 INFO:teuthology.orchestra.run.smithi031.stdout:5514 write 0x25988f5 thru 0x259ee8a (0x6596 bytes) 2024-09-16T22:17:34.169 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.170 INFO:teuthology.orchestra.run.smithi031.stdout:5516 writesame 0x80f5303 thru 0x8101a97 data_size 0x15(0xc795 bytes) 2024-09-16T22:17:34.174 INFO:teuthology.orchestra.run.smithi031.stdout:5518 punch from 0x336d6fe to 0x3378f75, (0xb877 bytes) 2024-09-16T22:17:34.175 INFO:teuthology.orchestra.run.smithi031.stdout:5519 writesame 0x964b457 thru 0x964f192 data_size 0x2(0x3d3c bytes) 2024-09-16T22:17:34.179 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:34.180+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:34.180 INFO:teuthology.orchestra.run.smithi031.stdout:5520 read 0x59cba5a thru 0x59cee16 (0x33bd bytes) 2024-09-16T22:17:34.187 INFO:teuthology.orchestra.run.smithi031.stdout:5521 write 0xabbb3f9 thru 0xabc1043 (0x5c4b bytes) 2024-09-16T22:17:34.191 INFO:teuthology.orchestra.run.smithi031.stdout:5525 write 0x1d5d8a5 thru 0x1d6d25e (0xf9ba bytes) 2024-09-16T22:17:34.201 INFO:teuthology.orchestra.run.smithi031.stdout:5526 write 0xccb8f68 thru 0xccbc0df (0x3178 bytes) 2024-09-16T22:17:34.204 INFO:teuthology.orchestra.run.smithi031.stdout:5527 write 0xb6a68dd thru 0xb6af043 (0x8767 bytes) 2024-09-16T22:17:34.207 INFO:teuthology.orchestra.run.smithi031.stdout:5528 write 0x6d3174d thru 0x6d4099b (0xf24f bytes) 2024-09-16T22:17:34.211 INFO:teuthology.orchestra.run.smithi031.stdout:5530 read 0x67091e9 thru 0x671661b (0xd433 bytes) 2024-09-16T22:17:34.211 INFO:teuthology.orchestra.run.smithi031.stdout:5531 punch from 0x1ca1c71 to 0x1ca4b79, (0x2f08 bytes) 2024-09-16T22:17:34.211 INFO:teuthology.orchestra.run.smithi031.stdout:5532 write 0x10b8822 thru 0x10c2bcc (0xa3ab bytes) 2024-09-16T22:17:34.219 INFO:teuthology.orchestra.run.smithi031.stdout:5533 write 0xcff1445 thru 0xcffb829 (0xa3e5 bytes) 2024-09-16T22:17:34.222 INFO:teuthology.orchestra.run.smithi031.stdout:5534 writesame 0xda6bd2 thru 0xdb351d data_size 0x1a(0xc94c bytes) 2024-09-16T22:17:34.225 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.225 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:34.238 INFO:teuthology.orchestra.run.smithi031.stdout:5535 trunc from 0xcffb82a to 0xcd64ee 2024-09-16T22:17:34.253 INFO:teuthology.orchestra.run.smithi031.stdout:5536 punch from 0x7aa7e8 to 0x7b5971, (0xb189 bytes) 2024-09-16T22:17:34.253 INFO:teuthology.orchestra.run.smithi031.stdout:5537 punch from 0x459f58 to 0x459f9b, (0x43 bytes) 2024-09-16T22:17:34.265 INFO:teuthology.orchestra.run.smithi031.stdout:5538 write 0x62cfe72 thru 0x62d519c (0x532b bytes) 2024-09-16T22:17:34.267 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.267 INFO:teuthology.orchestra.run.smithi031.stdout:5541 punch from 0x28e687b to 0x28ed4e1, (0x6c66 bytes) 2024-09-16T22:17:34.275 INFO:teuthology.orchestra.run.smithi031.stdout:5543 writesame 0x7f2ba7e thru 0x7f31f17 data_size 0x4f(0x649a bytes) 2024-09-16T22:17:34.275 INFO:teuthology.orchestra.run.smithi031.stdout:5545 read 0x3a748f1 thru 0x3a7df36 (0x9646 bytes) 2024-09-16T22:17:34.275 INFO:teuthology.orchestra.run.smithi031.stdout:5546 writesame 0x687e794 thru 0x687f7b4 data_size 0x1(0x1021 bytes) 2024-09-16T22:17:34.278 INFO:teuthology.orchestra.run.smithi031.stdout:5547 clone 44 order 18 su 2048 sc 11 2024-09-16T22:17:34.279 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:34.281+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:34.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:17:34.379 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:34.381+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:34.380 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.380 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:34.407 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone43 from 0x7f31f18 (overlap 0xcd64ee) to 0x52285f 2024-09-16T22:17:34.486 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:34.482+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:34.489 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-16T22:17:34.537 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.538 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:5548 trunc from 0x7f31f18 to 0xcfe60e0 2024-09-16T22:17:34.574 INFO:teuthology.orchestra.run.smithi031.stdout:5549 write 0x327896e thru 0x328699a (0xe02d bytes) 2024-09-16T22:17:34.587 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:34.589+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:34.660 INFO:teuthology.orchestra.run.smithi031.stdout:5550 trunc from 0xcfe60e0 to 0x87e27c6 2024-09-16T22:17:34.672 INFO:teuthology.orchestra.run.smithi031.stdout:5552 write 0x98d0eab thru 0x98d3095 (0x21eb bytes) 2024-09-16T22:17:34.682 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:34.684+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:34.683 INFO:teuthology.orchestra.run.smithi031.stdout:5554 punch from 0x52efa9d to 0x52f8160, (0x86c3 bytes) 2024-09-16T22:17:34.683 INFO:teuthology.orchestra.run.smithi031.stdout:5555 read 0x130e4c8 thru 0x1314953 (0x648c bytes) 2024-09-16T22:17:34.685 INFO:teuthology.orchestra.run.smithi031.stdout:5556 read 0x24ba2b3 thru 0x24c822a (0xdf78 bytes) 2024-09-16T22:17:34.693 INFO:teuthology.orchestra.run.smithi031.stdout:5558 writesame 0x9f9c80d thru 0x9fa0b62 data_size 0x33(0x4356 bytes) 2024-09-16T22:17:34.703 INFO:teuthology.orchestra.run.smithi031.stdout:5559 trunc from 0x9fa0b63 to 0x13a6ea8 2024-09-16T22:17:34.707 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.708 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:34.775 INFO:teuthology.orchestra.run.smithi031.stdout:5562 write 0x5a458ab thru 0x5a5520c (0xf962 bytes) 2024-09-16T22:17:34.794 INFO:teuthology.orchestra.run.smithi031.stdout:5563 writesame 0x1139add thru 0x113c479 data_size 0x35(0x299d bytes) 2024-09-16T22:17:34.805 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:34.807+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:34.828 INFO:teuthology.orchestra.run.smithi031.stdout:5565 write 0x3576d78 thru 0x3582061 (0xb2ea bytes) 2024-09-16T22:17:34.853 INFO:teuthology.orchestra.run.smithi031.stdout:5566 writesame 0x73c0491 thru 0x73ca788 data_size 0x46(0xa2f8 bytes) 2024-09-16T22:17:34.853 INFO:teuthology.orchestra.run.smithi031.stdout:5567 read 0x217329b thru 0x2178647 (0x53ad bytes) 2024-09-16T22:17:34.859 INFO:teuthology.orchestra.run.smithi031.stdout:5568 write 0x7814f80 thru 0x781e2cb (0x934c bytes) 2024-09-16T22:17:34.875 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.875 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.875 INFO:teuthology.orchestra.run.smithi031.stdout:5571 punch from 0x3f61b80 to 0x3f6a7af, (0x8c2f bytes) 2024-09-16T22:17:34.876 INFO:teuthology.orchestra.run.smithi031.stdout:5573 punch from 0x261e4b4 to 0x262702f, (0x8b7b bytes) 2024-09-16T22:17:34.876 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.884 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:34.885+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:34.885 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:34.885 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:34.887 INFO:teuthology.orchestra.run.smithi031.stdout:5575 write 0xdf38e8e thru 0xdf3cd82 (0x3ef5 bytes) 2024-09-16T22:17:34.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:34.902 INFO:teuthology.orchestra.run.smithi031.stdout:5576 read 0x21b05b6 thru 0x21b6175 (0x5bc0 bytes) 2024-09-16T22:17:34.902 INFO:teuthology.orchestra.run.smithi031.stdout:5577 punch from 0x78699bb to 0x7877b4e, (0xe193 bytes) 2024-09-16T22:17:34.903 INFO:teuthology.orchestra.run.smithi031.stdout:5578 read 0x5f21c50 thru 0x5f23eef (0x22a0 bytes) 2024-09-16T22:17:34.903 INFO:teuthology.orchestra.run.smithi031.stdout:5579 write 0x1a55b93 thru 0x1a62856 (0xccc4 bytes) 2024-09-16T22:17:34.913 INFO:teuthology.orchestra.run.smithi031.stdout:5580 read 0x15e349b thru 0x15ef165 (0xbccb bytes) 2024-09-16T22:17:34.920 INFO:teuthology.orchestra.run.smithi031.stdout:5581 writesame 0xe749db7 thru 0xe7559b8 data_size 0x5(0xbc02 bytes) 2024-09-16T22:17:34.921 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.921 INFO:teuthology.orchestra.run.smithi031.stdout:5583 write 0x5d11c85 thru 0x5d1ecdf (0xd05b bytes) 2024-09-16T22:17:34.946 INFO:teuthology.orchestra.run.smithi031.stdout:5584 trunc from 0xe7559b9 to 0x1cc9beb 2024-09-16T22:17:34.969 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:34.969 INFO:teuthology.orchestra.run.smithi031.stdout:5586 read 0x1ad86fb thru 0x1ae117b (0x8a81 bytes) 2024-09-16T22:17:34.970 INFO:teuthology.orchestra.run.smithi031.stdout:5588 read 0x13f8d14 thru 0x13fd521 (0x480e bytes) 2024-09-16T22:17:34.970 INFO:teuthology.orchestra.run.smithi031.stdout:5589 read 0x35e00d thru 0x363590 (0x5584 bytes) 2024-09-16T22:17:34.972 INFO:teuthology.orchestra.run.smithi031.stdout:5590 read 0x6fef3f thru 0x7002d5 (0x1397 bytes) 2024-09-16T22:17:34.973 INFO:teuthology.orchestra.run.smithi031.stdout:5592 read 0x35c5d thru 0x3f931 (0x9cd5 bytes) 2024-09-16T22:17:34.985 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:34.986+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:34.999 INFO:teuthology.orchestra.run.smithi031.stdout:5593 write 0xdb5f042 thru 0xdb6e93e (0xf8fd bytes) 2024-09-16T22:17:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:5594 write 0x48a59cd thru 0x48aebf6 (0x922a bytes) 2024-09-16T22:17:35.024 INFO:teuthology.orchestra.run.smithi031.stdout:5595 write 0x69f86bc thru 0x69fee7c (0x67c1 bytes) 2024-09-16T22:17:35.031 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:35.031 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:35.035 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:35.035 INFO:teuthology.orchestra.run.smithi031.stdout:5598 write 0x9988b68 thru 0x99988d2 (0xfd6b bytes) 2024-09-16T22:17:35.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:35.046 INFO:teuthology.orchestra.run.smithi031.stdout:5599 write 0x83404dd thru 0x834bdd6 (0xb8fa bytes) 2024-09-16T22:17:35.061 INFO:teuthology.orchestra.run.smithi031.stdout:5601 punch from 0x6359a9d to 0x635cbbc, (0x311f bytes) 2024-09-16T22:17:35.061 INFO:teuthology.orchestra.run.smithi031.stdout:5602 read 0x75083ba thru 0x7512671 (0xa2b8 bytes) 2024-09-16T22:17:35.061 INFO:teuthology.orchestra.run.smithi031.stdout:5604 write 0x908bb3e thru 0x909b79b (0xfc5e bytes) 2024-09-16T22:17:35.075 INFO:teuthology.orchestra.run.smithi031.stdout:5605 punch from 0x8b0b008 to 0x8b19af7, (0xeaef bytes) 2024-09-16T22:17:35.076 INFO:teuthology.orchestra.run.smithi031.stdout:5607 punch from 0x420d31d to 0x420eaa5, (0x1788 bytes) 2024-09-16T22:17:35.076 INFO:teuthology.orchestra.run.smithi031.stdout:5609 writesame 0xa82342b thru 0xa8323e5 data_size 0x9(0xefbb bytes) 2024-09-16T22:17:35.086 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:35.087+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:35.096 INFO:teuthology.orchestra.run.smithi031.stdout:5610 writesame 0xb6ecd2a thru 0xb6f340a data_size 0x1(0x66e1 bytes) 2024-09-16T22:17:35.111 INFO:teuthology.orchestra.run.smithi031.stdout:5611 punch from 0x81dc5fc to 0x81e7da6, (0xb7aa bytes) 2024-09-16T22:17:35.111 INFO:teuthology.orchestra.run.smithi031.stdout:5612 read 0xabdb59c thru 0xabe6ff6 (0xba5b bytes) 2024-09-16T22:17:35.124 INFO:teuthology.orchestra.run.smithi031.stdout:5613 trunc from 0xdb6e93f to 0x24b3e5e 2024-09-16T22:17:35.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:35.156 INFO:teuthology.orchestra.run.smithi031.stdout:5616 read 0x22a4f89 thru 0x22b166c (0xc6e4 bytes) 2024-09-16T22:17:35.156 INFO:teuthology.orchestra.run.smithi031.stdout:5617 clone 45 order 22 su 65536 sc 14 2024-09-16T22:17:35.175 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:35.176 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:35.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:35.186 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:35.188+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:35.267 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:35.267 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:35.284 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:35.287 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:35.289+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:35.335 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.274518984, 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115259 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.253995243999999, 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043168221, 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115259 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082584+0000", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.082587+0000", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1980000000000001e-06 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087294+0000", 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047076490000000004 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087309+0000", 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4551e-05 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087318+0000", 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087372+0000", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4184999999999997e-05 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087492+0000", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119762 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.087887+0000", 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039564899999999998 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.124754+0000", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036866492000000001 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125690+0000", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093677799999999998 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125726+0000", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5160999999999998e-05 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.125752+0000", 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6177e-05 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131066 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 933888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616241245000001, 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035857077000000001, 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131066 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720338+0000", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.46e-07 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720339+0000", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0580000000000001e-06 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720343+0000", 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720346+0000", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4360000000000001e-06 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756115+0000", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035768988000000002 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756155+0000", 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0742999999999999e-05 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756195+0000", 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9270999999999997e-05 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131067 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 983040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616230085000002, 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036032743999999998, 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131067 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720349+0000", 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720350+0000", 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.063e-06 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720353+0000", 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9390000000000002e-06 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720356+0000", 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5289999999999998e-06 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756226+0000", 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035870446 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756303+0000", 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6602999999999998e-05 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756382+0000", 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8951999999999999e-05 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131068 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1032192~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616220214999998, 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036117121000000002, 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131068 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720359+0000", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720360+0000", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720363+0000", 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720365+0000", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4719999999999998e-06 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756397+0000", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036031615000000003 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756437+0000", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0006999999999998e-05 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756476+0000", 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8804999999999999e-05 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131101 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1196032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616210505000002, 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036281198000000001, 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131101 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720368+0000", 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720369+0000", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720370+0000", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.023e-06 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720373+0000", 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720375+0000", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756508+0000", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036133086000000002 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756577+0000", 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9148999999999998e-05 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756650+0000", 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2290000000000001e-05 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131102 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1245184~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616198833999999, 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036372546999999998, 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131102 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720380+0000", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720382+0000", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7120000000000001e-06 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720388+0000", 2024-09-16T22:17:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5720000000000002e-06 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720398+0000", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0433e-05 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756677+0000", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036278794000000003 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756716+0000", 2024-09-16T22:17:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9436e-05 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756753+0000", 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6282000000000003e-05 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131118 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1294336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.61617438, 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036544986000000002, 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131118 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720404+0000", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720405+0000", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720407+0000", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0159999999999998e-06 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720413+0000", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6779999999999997e-06 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720417+0000", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4909999999999999e-06 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756785+0000", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036368075 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756867+0000", 2024-09-16T22:17:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1853000000000004e-05 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756949+0000", 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2453000000000005e-05 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131171 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1056768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616054435999999, 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036546147000000001, 2024-09-16T22:17:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131171 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720524+0000", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720525+0000", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720526+0000", 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.029e-06 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720530+0000", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-06 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720533+0000", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.756986+0000", 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036453754999999997 2024-09-16T22:17:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757032+0000", 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5531999999999997e-05 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757071+0000", 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8751000000000002e-05 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131172 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1105920~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616043073, 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036728516000000003, 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131172 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720536+0000", 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720537+0000", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720540+0000", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720543+0000", 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757109+0000", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036566047999999997 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757196+0000", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7019000000000002e-05 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757264+0000", 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8374000000000006e-05 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131173 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1155072~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.616032925999999, 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036824200000000001, 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131173 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720546+0000", 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-07 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720547+0000", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0979999999999999e-06 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720550+0000", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.129e-06 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720553+0000", 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.492e-06 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757280+0000", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036726795 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757329+0000", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9703000000000002e-05 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757370+0000", 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0748999999999999e-05 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131174 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1204224~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615998747999999, 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036974039, 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131174 2024-09-16T22:17:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720580+0000", 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720582+0000", 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.531e-06 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720586+0000", 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8190000000000002e-06 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720589+0000", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8040000000000002e-06 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757402+0000", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036813564999999999 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757480+0000", 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8103999999999996e-05 2024-09-16T22:17:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757554+0000", 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3825999999999994e-05 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131175 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1253376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615986037999999, 2024-09-16T22:17:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037074818000000002, 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131175 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720593+0000", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3000000000000002e-07 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720594+0000", 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.035e-06 2024-09-16T22:17:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720597+0000", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720600+0000", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757586+0000", 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036985393999999998 2024-09-16T22:17:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757628+0000", 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2644e-05 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757668+0000", 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9307000000000002e-05 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131176 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:23 [read 1302528~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615975464000002, 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037249833000000003, 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131176 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720603+0000", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720604+0000", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720605+0000", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7199999999999997e-07 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720608+0000", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.869e-06 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720610+0000", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757701+0000", 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037091164000000003 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757780+0000", 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.852e-05 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757853+0000", 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3368999999999995e-05 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131177 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1089536~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615965585000001, 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037365571, 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131177 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720613+0000", 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720615+0000", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.034e-06 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720617+0000", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720620+0000", 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-16T22:17:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757885+0000", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037265057999999997 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757935+0000", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0112999999999999e-05 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.757979+0000", 2024-09-16T22:17:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3584000000000001e-05 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131178 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1138688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615955687, 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037553098, 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131178 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720623+0000", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3099999999999999e-07 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720630+0000", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4300000000000003e-06 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720633+0000", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2160000000000002e-06 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720636+0000", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758017+0000", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037380913000000002 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758097+0000", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0029000000000002e-05 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758176+0000", 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9716999999999996e-05 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131179 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1187840~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615939752999999, 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037658184999999997, 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131179 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720639+0000", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999998e-07 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720640+0000", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.037e-06 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720644+0000", 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0469999999999998e-06 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720646+0000", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5009999999999999e-06 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758204+0000", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037558084999999998 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758254+0000", 2024-09-16T22:17:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9777000000000001e-05 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758297+0000", 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3405000000000002e-05 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131180 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1236992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615929817000001, 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037835806, 2024-09-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131180 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720649+0000", 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720650+0000", 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031e-06 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720653+0000", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.892e-06 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720656+0000", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.339e-06 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758335+0000", 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037679556000000003 2024-09-16T22:17:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758409+0000", 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3980000000000007e-05 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758485+0000", 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5797999999999995e-05 2024-09-16T22:17:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:131181 3.f 3:f62eb465:::rbd_data.11cd289e90a9.000000000000000b:23 [read 1286144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.615920318000001, 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037949063999999998, 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131181 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720659+0000", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4299999999999999e-07 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720660+0000", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720663+0000", 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.720665+0000", 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758515+0000", 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037850097999999999 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758564+0000", 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8179000000000001e-05 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:16.758608+0000", 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4088000000000002e-05 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6862948420000001, 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:35.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:35.426 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:17:35.427 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone44 from 0x24b3e5e (overlap 0x13a6ea8) to 0x0 2024-09-16T22:17:35.427 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:35.427+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:35.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:35.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:35.525 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:35.527+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:35.565 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:35.565 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:35.593 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:35.594 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:35.626 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:35.626 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:35.627 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:35.628+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:35.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066819139999999999, 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066833770000000001, 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:35.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061506916000000002, 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061553522999999999, 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:17:35.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195879 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594154+0000", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.054978051, 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055050489000000001, 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195879 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594154+0000", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594154+0000", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594154+0000", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9800000000000004e-07 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594156+0000", 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-06 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594166+0000", 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3589999999999997e-06 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594172+0000", 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196011 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604608+0000", 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044523353000000002, 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044620130000000001, 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196011 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604608+0000", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604608+0000", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604609+0000", 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-09-16T22:17:35.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604611+0000", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1449999999999998e-06 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604618+0000", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7889999999999997e-06 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604623+0000", 2024-09-16T22:17:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.198e-06 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196144 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.611290+0000", 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037841821999999997, 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037961978, 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196144 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611290+0000", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611290+0000", 2024-09-16T22:17:35.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611290+0000", 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5999999999999999e-07 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611292+0000", 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.295e-06 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611299+0000", 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2400000000000004e-06 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611304+0000", 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8110000000000002e-06 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196275 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.623144+0000", 2024-09-16T22:17:35.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025987967000000001, 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026133117000000001, 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196275 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623144+0000", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623144+0000", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623144+0000", 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6699999999999999e-07 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623146+0000", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.933e-06 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623153+0000", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0609999999999998e-06 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623159+0000", 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4650000000000002e-06 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196406 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631375+0000", 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017756568, 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017925098, 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196406 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631375+0000", 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631375+0000", 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631376+0000", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5600000000000005e-07 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631380+0000", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9450000000000003e-06 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631417+0000", 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.769e-05 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631425+0000", 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066796487000000002, 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066989461, 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:17:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061487435, 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703702999999999, 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195880 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594179+0000", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.054952246000000003, 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055193711999999999, 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195880 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594179+0000", 2024-09-16T22:17:35.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594179+0000", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594180+0000", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-07 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594182+0000", 2024-09-16T22:17:35.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.08e-06 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594189+0000", 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9650000000000002e-06 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594194+0000", 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196012 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604630+0000", 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044502027999999999, 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044777892999999999, 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196012 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604630+0000", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604630+0000", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604630+0000", 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604632+0000", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604639+0000", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.866e-06 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604644+0000", 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196407 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631438+0000", 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017693963, 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017994759999999999, 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196407 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631438+0000", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631438+0000", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631438+0000", 2024-09-16T22:17:35.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8999999999999996e-07 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631442+0000", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3699999999999999e-06 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631449+0000", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.216e-06 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631454+0000", 2024-09-16T22:17:35.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5829999999999997e-06 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066773508999999995, 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067098574999999994, 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061463095000000002, 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061811449999999997, 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:17:35.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195881 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594201+0000", 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.054930824000000003, 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055302389, 2024-09-16T22:17:35.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195881 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594201+0000", 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594201+0000", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594201+0000", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594203+0000", 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0880000000000002e-06 2024-09-16T22:17:35.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594210+0000", 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4699999999999999e-06 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594215+0000", 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8520000000000003e-06 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196013 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604652+0000", 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044479946999999999, 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044874954000000002, 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196013 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604652+0000", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604652+0000", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604652+0000", 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604654+0000", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604661+0000", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604667+0000", 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5090000000000001e-06 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066750456, 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067170831, 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:17:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061443128999999999, 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061890161999999999, 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195882 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594221+0000", 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.054910754999999999, 2024-09-16T22:17:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055381576000000002, 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195882 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594221+0000", 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594221+0000", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594221+0000", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594223+0000", 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594229+0000", 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.959e-06 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594234+0000", 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7679999999999999e-06 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196014 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604674+0000", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044457919999999998, 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044952251999999998, 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196014 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604674+0000", 2024-09-16T22:17:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604674+0000", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604674+0000", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604676+0000", 2024-09-16T22:17:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.886e-06 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604682+0000", 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3779999999999998e-06 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604687+0000", 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6029999999999998e-06 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066728735999999997, 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067246712, 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061421718, 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061963569000000003, 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:35.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:17:35.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066708532000000001, 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067273876999999996, 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061399956999999998, 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061988749000000003, 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:35.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:17:35.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066689941000000003, 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067302449, 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:17:35.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061379489000000002, 2024-09-16T22:17:35.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062015467999999997, 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195462 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 180224~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581635+0000", 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067496374999999997, 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068164393000000004, 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195462 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581635+0000", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581635+0000", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581636+0000", 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581638+0000", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.294e-06 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581645+0000", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7159999999999997e-06 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581649+0000", 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.649663+0000", 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068013052000000004 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.649755+0000", 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2938999999999999e-05 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066668410999999997, 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067364553999999993, 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:35.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195463 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581659+0000", 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067472509, 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068191964999999993, 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195463 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581659+0000", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581659+0000", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581660+0000", 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581662+0000", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3140000000000002e-06 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581669+0000", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581674+0000", 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1200000000000001e-06 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195464 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581681+0000", 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067450129999999997, 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068192938999999994, 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195464 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581681+0000", 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581681+0000", 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581682+0000", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581684+0000", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581691+0000", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5729999999999996e-06 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581696+0000", 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.395e-06 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195465 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581717+0000", 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067414247999999996, 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068180550000000006, 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195465 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581717+0000", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581717+0000", 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581718+0000", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581720+0000", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4870000000000001e-06 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581727+0000", 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9179999999999997e-06 2024-09-16T22:17:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581732+0000", 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2000000000000002e-06 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195466 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581769+0000", 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067362237000000005, 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068157803000000003, 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195466 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581769+0000", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581769+0000", 2024-09-16T22:17:35.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581770+0000", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0800000000000004e-07 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581774+0000", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.912e-06 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581787+0000", 2024-09-16T22:17:35.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2656e-05 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581795+0000", 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1529999999999997e-06 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195467 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581807+0000", 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067324247000000004, 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068144404000000006, 2024-09-16T22:17:35.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195467 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581807+0000", 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581807+0000", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581808+0000", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2399999999999998e-07 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581811+0000", 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7039999999999999e-06 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581818+0000", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.486e-06 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581824+0000", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195468 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581832+0000", 2024-09-16T22:17:35.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067299188999999995, 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068142995999999997, 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195468 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581832+0000", 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581832+0000", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581833+0000", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581835+0000", 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3530000000000002e-06 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581842+0000", 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5599999999999999e-06 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581847+0000", 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2410000000000003e-06 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195469 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581861+0000", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067270354000000004, 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068151823, 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195469 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581861+0000", 2024-09-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581861+0000", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581862+0000", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581864+0000", 2024-09-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9350000000000001e-06 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581876+0000", 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2795e-05 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581882+0000", 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196523 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640844+0000", 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082876679999999998, 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091956840000000008, 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196523 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640844+0000", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640844+0000", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640845+0000", 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8100000000000002e-07 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640849+0000", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1200000000000004e-06 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640859+0000", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0655e-05 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640866+0000", 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8170000000000001e-06 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195470 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581902+0000", 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067229563000000006, 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068163089999999996, 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195470 2024-09-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581902+0000", 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581902+0000", 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581903+0000", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4000000000000002e-07 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581905+0000", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.39e-06 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581914+0000", 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581925+0000", 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0334e-05 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196127 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610628+0000", 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038503215, 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039460546999999999, 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196127 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610628+0000", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610628+0000", 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610629+0000", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6899999999999996e-07 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610634+0000", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4220000000000002e-06 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610662+0000", 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7886999999999999e-05 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610669+0000", 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5310000000000004e-06 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196524 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640878+0000", 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082533350000000005, 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092380569999999992, 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196524 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640878+0000", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640878+0000", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640879+0000", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640881+0000", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640888+0000", 2024-09-16T22:17:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4680000000000003e-06 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640894+0000", 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9170000000000003e-06 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195471 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581934+0000", 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067197204999999996, 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068205550000000004, 2024-09-16T22:17:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195471 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581934+0000", 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581934+0000", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581935+0000", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581937+0000", 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6120000000000001e-06 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581945+0000", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.593e-06 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581953+0000", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5850000000000002e-06 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196128 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610683+0000", 2024-09-16T22:17:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038448061999999998, 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039481416999999998, 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196128 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610683+0000", 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610683+0000", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610684+0000", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9500000000000002e-07 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610687+0000", 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3629999999999998e-06 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610695+0000", 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7919999999999993e-06 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610701+0000", 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5999999999999997e-06 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196259 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.621640+0000", 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.027491097999999999, 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028548191000000001, 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196259 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621640+0000", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621640+0000", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621641+0000", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.75e-07 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621648+0000", 2024-09-16T22:17:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.109e-06 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621933+0000", 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000285056 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621940+0000", 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999996e-06 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196525 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640902+0000", 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082296179999999993, 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093242499999999992, 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196525 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640902+0000", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640902+0000", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640902+0000", 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640904+0000", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0329999999999998e-06 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640911+0000", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9410000000000001e-06 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640917+0000", 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.288e-06 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195579 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581960+0000", 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067171460000000002, 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068318723999999997, 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195579 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581960+0000", 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581960+0000", 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581961+0000", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581963+0000", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2110000000000001e-06 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581971+0000", 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3969999999999994e-06 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581976+0000", 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3530000000000002e-06 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196129 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610710+0000", 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038421762999999998, 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039616292999999997, 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196129 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610710+0000", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610710+0000", 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610710+0000", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610713+0000", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.475e-06 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610719+0000", 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.742e-06 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610725+0000", 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2909999999999998e-06 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196260 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.621953+0000", 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.027178673, 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028427125000000001, 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196260 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621953+0000", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621953+0000", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621954+0000", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6000000000000003e-07 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621957+0000", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5319999999999998e-06 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621965+0000", 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7589999999999997e-06 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.621973+0000", 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9240000000000007e-06 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196391 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.630946+0000", 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018185153999999999, 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019484611999999998, 2024-09-16T22:17:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196391 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630946+0000", 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630946+0000", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630947+0000", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630949+0000", 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4739999999999999e-06 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630955+0000", 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630961+0000", 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2329999999999997e-06 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196526 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640932+0000", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081991700000000004, 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095508920000000001, 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196526 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640932+0000", 2024-09-16T22:17:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640932+0000", 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640933+0000", 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640935+0000", 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.384e-06 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640942+0000", 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0890000000000002e-06 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640948+0000", 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3709999999999999e-06 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195580 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.581984+0000", 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067147760000000001, 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068549772999999994, 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195580 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581984+0000", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581984+0000", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581984+0000", 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-09-16T22:17:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.581991+0000", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582000+0000", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3180000000000007e-06 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582005+0000", 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3159999999999996e-06 2024-09-16T22:17:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195997 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603447+0000", 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045685000000000003, 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047138177000000003, 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195997 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603447+0000", 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603447+0000", 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603447+0000", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603449+0000", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603456+0000", 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5520000000000001e-06 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603461+0000", 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196130 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610732+0000", 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038399243, 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039901461999999999, 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196130 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610732+0000", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610732+0000", 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610733+0000", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610735+0000", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-06 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610742+0000", 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7800000000000003e-06 2024-09-16T22:17:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610747+0000", 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0930000000000002e-06 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196261 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622703+0000", 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026428292999999999, 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027981539999999999, 2024-09-16T22:17:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196261 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622703+0000", 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622703+0000", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622704+0000", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622708+0000", 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9190000000000001e-06 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622719+0000", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1294000000000001e-05 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622727+0000", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7649999999999993e-06 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196392 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.630968+0000", 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018163862999999999, 2024-09-16T22:17:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019766389999999998, 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196392 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630968+0000", 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630968+0000", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630968+0000", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630970+0000", 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0389999999999999e-06 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630976+0000", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7100000000000004e-06 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630981+0000", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196527 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640955+0000", 2024-09-16T22:17:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081767909999999992, 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098301670000000008, 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196527 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640955+0000", 2024-09-16T22:17:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640955+0000", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640955+0000", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640957+0000", 2024-09-16T22:17:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.933e-06 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640964+0000", 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4289999999999998e-06 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640969+0000", 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195581 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582013+0000", 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067118472999999998, 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068833777999999998, 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195581 2024-09-16T22:17:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582013+0000", 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582013+0000", 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582014+0000", 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582016+0000", 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2340000000000001e-06 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582024+0000", 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2770000000000005e-06 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582029+0000", 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3800000000000002e-06 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195866 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593824+0000", 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055307338999999997, 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057075641000000003, 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195866 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593824+0000", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593824+0000", 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593825+0000", 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593827+0000", 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593833+0000", 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.246e-06 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593838+0000", 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8539999999999996e-06 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195998 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603468+0000", 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0456639, 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047487033999999997, 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195998 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603468+0000", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603468+0000", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603468+0000", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603470+0000", 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0020000000000001e-06 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603476+0000", 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3130000000000004e-06 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603481+0000", 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8230000000000002e-06 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196131 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610755+0000", 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038376588000000003, 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.040256878000000003, 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196131 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610755+0000", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610755+0000", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610756+0000", 2024-09-16T22:17:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-07 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610758+0000", 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610764+0000", 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610769+0000", 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9749999999999998e-06 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196262 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622739+0000", 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026392632999999999, 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028331358000000001, 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196262 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622739+0000", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622739+0000", 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622739+0000", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622742+0000", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622751+0000", 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1509999999999993e-06 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622758+0000", 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7819999999999996e-06 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196393 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.630987+0000", 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018144282000000001, 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020137366, 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196393 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630987+0000", 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630987+0000", 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630988+0000", 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630990+0000", 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9410000000000001e-06 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.630996+0000", 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631001+0000", 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8350000000000003e-06 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196528 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640975+0000", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081563090000000005, 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010202466, 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196528 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640975+0000", 2024-09-16T22:17:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640975+0000", 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640976+0000", 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640978+0000", 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0499999999999999e-06 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640984+0000", 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7819999999999999e-06 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640988+0000", 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5940000000000004e-06 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195582 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582036+0000", 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067095826999999997, 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069189562999999996, 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195582 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582036+0000", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582036+0000", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582036+0000", 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.65e-07 2024-09-16T22:17:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582038+0000", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.956e-06 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582044+0000", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0950000000000001e-06 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582049+0000", 2024-09-16T22:17:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195867 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593845+0000", 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055286758999999998, 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057432637000000002, 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195867 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593845+0000", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593845+0000", 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593845+0000", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593847+0000", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2419999999999999e-06 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593854+0000", 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4350000000000002e-06 2024-09-16T22:17:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593859+0000", 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9509999999999997e-06 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195999 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603488+0000", 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045643481, 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047834238000000001, 2024-09-16T22:17:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195999 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603488+0000", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603488+0000", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603488+0000", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603491+0000", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0789999999999999e-06 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603497+0000", 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2609999999999999e-06 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603502+0000", 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196132 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610776+0000", 2024-09-16T22:17:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038355470000000003, 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.040597162999999999, 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196132 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610776+0000", 2024-09-16T22:17:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610776+0000", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610777+0000", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610779+0000", 2024-09-16T22:17:35.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610785+0000", 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0839999999999997e-06 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610790+0000", 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9960000000000001e-06 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196263 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622768+0000", 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026363775999999998, 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028653873999999999, 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196263 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622768+0000", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622768+0000", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622768+0000", 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-07 2024-09-16T22:17:35.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622771+0000", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8200000000000001e-06 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622779+0000", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5730000000000002e-06 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622784+0000", 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6509999999999998e-06 2024-09-16T22:17:35.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196394 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631007+0000", 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018124561000000001, 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020466053000000001, 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196394 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631007+0000", 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631007+0000", 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631007+0000", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631009+0000", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631017+0000", 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631022+0000", 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2610000000000001e-06 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196529 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.640996+0000", 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081358899999999998, 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010526146, 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196529 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640996+0000", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640996+0000", 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640996+0000", 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.640998+0000", 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2019999999999998e-06 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641005+0000", 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9949999999999999e-06 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641010+0000", 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195583 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582055+0000", 2024-09-16T22:17:35.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067076853000000006, 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06953173, 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195583 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582055+0000", 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582055+0000", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582055+0000", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582057+0000", 2024-09-16T22:17:35.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7460000000000001e-06 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582064+0000", 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.396e-06 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582070+0000", 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4639999999999997e-06 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195868 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593865+0000", 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055266577999999997, 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057770237000000002, 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195868 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593865+0000", 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593865+0000", 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593865+0000", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593868+0000", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.154e-06 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593873+0000", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8719999999999999e-06 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593878+0000", 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7740000000000004e-06 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196000 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603508+0000", 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045623710999999997, 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048174889999999998, 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196000 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603508+0000", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603508+0000", 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603508+0000", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.58e-07 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603510+0000", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8449999999999999e-06 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603516+0000", 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-16T22:17:35.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603521+0000", 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196133 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610796+0000", 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038335645000000002, 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.040933144999999997, 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196133 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610796+0000", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610796+0000", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610796+0000", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-07 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610798+0000", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9889999999999999e-06 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610804+0000", 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8710000000000002e-06 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610809+0000", 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6689999999999997e-06 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196264 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622793+0000", 2024-09-16T22:17:35.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026338704000000001, 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028986442000000001, 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196264 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622793+0000", 2024-09-16T22:17:35.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622793+0000", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622793+0000", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622796+0000", 2024-09-16T22:17:35.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622803+0000", 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7719999999999997e-06 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622808+0000", 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-06 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196395 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631029+0000", 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018102285999999999, 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020798223000000001, 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196395 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631029+0000", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631029+0000", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631030+0000", 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-09-16T22:17:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631032+0000", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631038+0000", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631043+0000", 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7190000000000001e-06 2024-09-16T22:17:35.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196530 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641018+0000", 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081138180000000001, 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010857311, 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196530 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641018+0000", 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641018+0000", 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641018+0000", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-07 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641020+0000", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0420000000000001e-06 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641033+0000", 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2369e-05 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641038+0000", 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6799999999999998e-06 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195584 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582075+0000", 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067056183000000005, 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069855832000000007, 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195584 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582075+0000", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582075+0000", 2024-09-16T22:17:35.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582076+0000", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582078+0000", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9309999999999998e-06 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582085+0000", 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3549999999999999e-06 2024-09-16T22:17:35.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582090+0000", 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6439999999999999e-06 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195734 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587358+0000", 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061773282999999998, 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064621607999999997, 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195734 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587358+0000", 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587358+0000", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587359+0000", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2200000000000004e-07 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587361+0000", 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4619999999999999e-06 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587368+0000", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1749999999999999e-06 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587374+0000", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4650000000000002e-06 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195869 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593885+0000", 2024-09-16T22:17:35.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055246638000000001, 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058142955000000003, 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195869 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593885+0000", 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593885+0000", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593885+0000", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593887+0000", 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0329999999999998e-06 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593894+0000", 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.117e-06 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593900+0000", 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196001 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603527+0000", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045604057000000003, 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048548833, 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196001 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603527+0000", 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603527+0000", 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603528+0000", 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603530+0000", 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603536+0000", 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9390000000000002e-06 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603540+0000", 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5730000000000001e-06 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196134 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610815+0000", 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038316280000000001, 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.041308588, 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196134 2024-09-16T22:17:35.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610815+0000", 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610815+0000", 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610816+0000", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610818+0000", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610824+0000", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610829+0000", 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7729999999999999e-06 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196265 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622817+0000", 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026314766999999999, 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029355249999999999, 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196265 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622817+0000", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622817+0000", 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622817+0000", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-07 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622820+0000", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.402e-06 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622827+0000", 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1890000000000001e-06 2024-09-16T22:17:35.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622833+0000", 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196396 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631049+0000", 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018082514000000001, 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021170978, 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196396 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631049+0000", 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631049+0000", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631049+0000", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2199999999999999e-07 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631051+0000", 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-06 2024-09-16T22:17:35.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631058+0000", 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2360000000000001e-06 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631064+0000", 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1229999999999997e-06 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196531 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641047+0000", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080848540000000007, 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011224088, 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196531 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641047+0000", 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641047+0000", 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641047+0000", 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641050+0000", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.559e-06 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641057+0000", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641062+0000", 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9790000000000001e-06 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195585 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582096+0000", 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067035548, 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070224403000000005, 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195585 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582096+0000", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582096+0000", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582096+0000", 2024-09-16T22:17:35.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582099+0000", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582107+0000", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1210000000000007e-06 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582112+0000", 2024-09-16T22:17:35.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195735 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587425+0000", 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061706916000000001, 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064942825999999995, 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195735 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587425+0000", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587425+0000", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587425+0000", 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587428+0000", 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587436+0000", 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1040000000000007e-06 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587442+0000", 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9549999999999997e-06 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195870 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593907+0000", 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055224154999999997, 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058508975999999997, 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195870 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593907+0000", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593907+0000", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593908+0000", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593910+0000", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593916+0000", 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2750000000000001e-06 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593930+0000", 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4078e-05 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196002 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603547+0000", 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045584073000000003, 2024-09-16T22:17:35.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049212921999999999, 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196002 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603547+0000", 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603547+0000", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603548+0000", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603550+0000", 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-16T22:17:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603557+0000", 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603562+0000", 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196135 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610834+0000", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038297114, 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.041987253000000002, 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196135 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610834+0000", 2024-09-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610834+0000", 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610835+0000", 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610837+0000", 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610843+0000", 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2360000000000001e-06 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610848+0000", 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196266 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622841+0000", 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026290375000000001, 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030030298, 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196266 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622841+0000", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622841+0000", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622842+0000", 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622844+0000", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622851+0000", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3039999999999999e-06 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622857+0000", 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196397 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631072+0000", 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018059664999999999, 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021858081000000001, 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196397 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631072+0000", 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631072+0000", 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631072+0000", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631074+0000", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.057e-06 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631081+0000", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631086+0000", 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196532 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641069+0000", 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080625599999999999, 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011910250000000001, 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196532 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641069+0000", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641069+0000", 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641069+0000", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641072+0000", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641078+0000", 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8730000000000001e-06 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641084+0000", 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.113e-06 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195586 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582120+0000", 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.067012007999999998, 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070908451999999997, 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195586 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582120+0000", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582120+0000", 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582120+0000", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582122+0000", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582131+0000", 2024-09-16T22:17:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3899999999999993e-06 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582137+0000", 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5339999999999996e-06 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195736 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587451+0000", 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061680711999999999, 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065625632000000003, 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195736 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587451+0000", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587451+0000", 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587451+0000", 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587454+0000", 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4770000000000002e-06 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587461+0000", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1709999999999996e-06 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587467+0000", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9880000000000001e-06 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195871 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593938+0000", 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055193278999999998, 2024-09-16T22:17:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059186038000000003, 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195871 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593938+0000", 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593938+0000", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593939+0000", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593941+0000", 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.401e-06 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593947+0000", 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2020000000000001e-06 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593952+0000", 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9150000000000004e-06 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196003 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603569+0000", 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045562549000000001, 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049601353000000001, 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196003 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603569+0000", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603569+0000", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603569+0000", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603571+0000", 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.956e-06 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603592+0000", 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0924000000000002e-05 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603598+0000", 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6890000000000001e-06 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196136 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610855+0000", 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038276115999999999, 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042364717000000003, 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196136 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610855+0000", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610855+0000", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610856+0000", 2024-09-16T22:17:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9699999999999996e-07 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610858+0000", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2579999999999998e-06 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610864+0000", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1739999999999997e-06 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610869+0000", 2024-09-16T22:17:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196267 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622863+0000", 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026268481, 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030404288000000002, 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196267 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622863+0000", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622863+0000", 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622863+0000", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622865+0000", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622872+0000", 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.635e-06 2024-09-16T22:17:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622878+0000", 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5929999999999996e-06 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196398 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631115+0000", 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.018016144000000001, 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022199362, 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196398 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631115+0000", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631115+0000", 2024-09-16T22:17:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631116+0000", 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631119+0000", 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7480000000000002e-06 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631134+0000", 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5211000000000001e-05 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631139+0000", 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3589999999999999e-06 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196533 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641090+0000", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080415639999999993, 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012273321, 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196533 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641090+0000", 2024-09-16T22:17:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641090+0000", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641090+0000", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641092+0000", 2024-09-16T22:17:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8920000000000001e-06 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641099+0000", 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929999999999996e-06 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641104+0000", 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0499999999999999e-06 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195587 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582145+0000", 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066986094999999996, 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071267643000000006, 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195587 2024-09-16T22:17:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582145+0000", 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582145+0000", 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582146+0000", 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582148+0000", 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1210000000000001e-06 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582156+0000", 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9699999999999999e-06 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582163+0000", 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195737 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587474+0000", 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061657650000000001, 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065987707000000007, 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195737 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587474+0000", 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587474+0000", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587474+0000", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587476+0000", 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587483+0000", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587488+0000", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195872 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593960+0000", 2024-09-16T22:17:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055171680000000001, 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059550445, 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195872 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593960+0000", 2024-09-16T22:17:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593960+0000", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593960+0000", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593963+0000", 2024-09-16T22:17:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593969+0000", 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5810000000000002e-06 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593974+0000", 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9180000000000002e-06 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196004 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603607+0000", 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045524321999999999, 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049953938000000003, 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196004 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603607+0000", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603607+0000", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603608+0000", 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603610+0000", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603618+0000", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1949999999999997e-06 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603623+0000", 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4179999999999996e-06 2024-09-16T22:17:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196137 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610877+0000", 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038254418999999998, 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042731863000000002, 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196137 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610877+0000", 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610877+0000", 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610878+0000", 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610880+0000", 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.39e-06 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610886+0000", 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1879999999999999e-06 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610891+0000", 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196268 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622884+0000", 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026247116000000001, 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030775950999999999, 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196268 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622884+0000", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622884+0000", 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622885+0000", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622887+0000", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.844e-06 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622894+0000", 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8510000000000001e-06 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622899+0000", 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1599999999999997e-06 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196399 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631149+0000", 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017983031999999999, 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022558491, 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196399 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631149+0000", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631149+0000", 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631149+0000", 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631152+0000", 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6539999999999999e-06 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631159+0000", 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9090000000000003e-06 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631165+0000", 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9909999999999999e-06 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196534 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641110+0000", 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080210890000000003, 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012645450000000001, 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196534 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641110+0000", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641110+0000", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641111+0000", 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641118+0000", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1810000000000004e-06 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641125+0000", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641131+0000", 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-06 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195588 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582171+0000", 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066960355999999999, 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071637528000000006, 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195588 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582171+0000", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582171+0000", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582172+0000", 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-16T22:17:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582174+0000", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1380000000000001e-06 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582181+0000", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.622e-06 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582188+0000", 2024-09-16T22:17:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3330000000000002e-06 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195738 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587495+0000", 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061636076999999997, 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066364326000000001, 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195738 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587495+0000", 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587495+0000", 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587496+0000", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587498+0000", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587505+0000", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9280000000000004e-06 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587511+0000", 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195873 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.593980+0000", 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055151057000000003, 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059929105000000003, 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195873 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593980+0000", 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593980+0000", 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593981+0000", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593983+0000", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0109999999999999e-06 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593990+0000", 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1289999999999998e-06 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.593995+0000", 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196005 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603630+0000", 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045501852000000002, 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050334858000000003, 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196005 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603630+0000", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603630+0000", 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603630+0000", 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603632+0000", 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.114e-06 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603641+0000", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603646+0000", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196138 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610897+0000", 2024-09-16T22:17:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038234187000000003, 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043117085999999999, 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196138 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610897+0000", 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610897+0000", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610898+0000", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610904+0000", 2024-09-16T22:17:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5710000000000003e-06 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610911+0000", 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.041e-06 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610917+0000", 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2979999999999999e-06 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196269 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622906+0000", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026225516000000001, 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031158642, 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196269 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622906+0000", 2024-09-16T22:17:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622906+0000", 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622906+0000", 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622908+0000", 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0030000000000001e-06 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622915+0000", 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8050000000000001e-06 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622930+0000", 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4494999999999999e-05 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196400 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631172+0000", 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017959468999999999, 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022939278, 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196400 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631172+0000", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631172+0000", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631172+0000", 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631175+0000", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631181+0000", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631186+0000", 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1479999999999997e-06 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196535 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641139+0000", 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079926149999999998, 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013021161, 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196535 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641139+0000", 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641139+0000", 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641139+0000", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641142+0000", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1459999999999999e-06 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641148+0000", 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641153+0000", 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2360000000000004e-06 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195589 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582196+0000", 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066935636000000007, 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072013630999999995, 2024-09-16T22:17:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195589 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582196+0000", 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582196+0000", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582196+0000", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582199+0000", 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582206+0000", 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582211+0000", 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0209999999999999e-06 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195739 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587518+0000", 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061613477, 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066739163000000004, 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195739 2024-09-16T22:17:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587518+0000", 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587518+0000", 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587518+0000", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587521+0000", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587527+0000", 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587532+0000", 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195874 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594002+0000", 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055129590999999999, 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060302712000000001, 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195874 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594002+0000", 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594002+0000", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594003+0000", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594005+0000", 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0660000000000002e-06 2024-09-16T22:17:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594011+0000", 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2349999999999996e-06 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594016+0000", 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196006 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603654+0000", 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045477823000000001, 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050699199, 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196006 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603654+0000", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603654+0000", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603654+0000", 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603657+0000", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.447e-06 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603663+0000", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603669+0000", 2024-09-16T22:17:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196139 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.610933+0000", 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.038198597000000001, 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043469010000000002, 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196139 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610933+0000", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610933+0000", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610933+0000", 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610936+0000", 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4760000000000001e-06 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610945+0000", 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.115e-06 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.610951+0000", 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196270 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622938+0000", 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026193162999999998, 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031511018000000002, 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196270 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622938+0000", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622938+0000", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622939+0000", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622941+0000", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622949+0000", 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5460000000000003e-06 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622954+0000", 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196401 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631193+0000", 2024-09-16T22:17:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017938938000000001, 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023305790999999999, 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196401 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631193+0000", 2024-09-16T22:17:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631193+0000", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631193+0000", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631195+0000", 2024-09-16T22:17:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.018e-06 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631201+0000", 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2820000000000002e-06 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631206+0000", 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9189999999999998e-06 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196536 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641161+0000", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079710030000000008, 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013386247, 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196536 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641161+0000", 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641161+0000", 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641161+0000", 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641163+0000", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641170+0000", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9870000000000002e-06 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641175+0000", 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195590 3.10 3.16c566f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582218+0000", 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066913893000000002, 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072379478999999997, 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195590 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582218+0000", 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582218+0000", 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582218+0000", 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582220+0000", 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.855e-06 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582227+0000", 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582232+0000", 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8659999999999997e-06 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195740 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587539+0000", 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061592257999999997, 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067105185999999997, 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195740 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587539+0000", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587539+0000", 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587540+0000", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587542+0000", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1270000000000001e-06 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587549+0000", 2024-09-16T22:17:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587554+0000", 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.152e-06 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195875 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594062+0000", 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055069941999999997, 2024-09-16T22:17:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060631302999999998, 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195875 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594062+0000", 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594062+0000", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594062+0000", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-07 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594065+0000", 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.18e-06 2024-09-16T22:17:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594073+0000", 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.943e-06 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594079+0000", 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5969999999999999e-06 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196007 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.603676+0000", 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.045455494999999999, 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051065135999999997, 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196007 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603676+0000", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603676+0000", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603677+0000", 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.603679+0000", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6180000000000002e-06 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604495+0000", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081591900000000002 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604505+0000", 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0122999999999999e-05 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196140 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.611198+0000", 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037933692999999997, 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043593390000000003, 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196140 2024-09-16T22:17:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611198+0000", 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611198+0000", 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611198+0000", 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611202+0000", 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3550000000000001e-06 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611211+0000", 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8559999999999996e-06 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611217+0000", 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196271 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.622961+0000", 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026170183999999999, 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031877541000000002, 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196271 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622961+0000", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622961+0000", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622962+0000", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622970+0000", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4379999999999995e-06 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.622978+0000", 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2439999999999996e-06 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623032+0000", 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4296999999999999e-05 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196402 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631213+0000", 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017918944999999999, 2024-09-16T22:17:36.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023674572000000001, 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196402 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631213+0000", 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631213+0000", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631213+0000", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631219+0000", 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1689999999999997e-06 2024-09-16T22:17:36.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631225+0000", 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631244+0000", 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8779000000000001e-05 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196537 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641182+0000", 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079498650000000004, 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013760667000000001, 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196537 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641182+0000", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641182+0000", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641182+0000", 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2300000000000002e-07 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641184+0000", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0719999999999998e-06 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641191+0000", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6540000000000002e-06 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641196+0000", 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2020000000000003e-06 2024-09-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066893959000000003, 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072753683, 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195741 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587560+0000", 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061571471000000003, 2024-09-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067479222000000005, 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195741 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587560+0000", 2024-09-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587560+0000", 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587560+0000", 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587562+0000", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.029e-06 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587569+0000", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587575+0000", 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3459999999999999e-06 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195876 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594088+0000", 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055043698000000002, 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060999587000000001, 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195876 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594088+0000", 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594088+0000", 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594088+0000", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594090+0000", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594097+0000", 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594102+0000", 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196008 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604526+0000", 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044605718000000003, 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050609670000000002, 2024-09-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196008 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604526+0000", 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604526+0000", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604526+0000", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-07 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604535+0000", 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3990000000000004e-06 2024-09-16T22:17:36.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604544+0000", 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5689999999999994e-06 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604551+0000", 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196141 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.611227+0000", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037904225, 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043956025000000003, 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196141 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611227+0000", 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611227+0000", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611228+0000", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611230+0000", 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.508e-06 2024-09-16T22:17:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611237+0000", 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611242+0000", 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0649999999999998e-06 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196272 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.623046+0000", 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026085560000000001, 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032334654999999997, 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196272 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623046+0000", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623046+0000", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623047+0000", 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9599999999999999e-07 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623050+0000", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.444e-06 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623058+0000", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0399999999999993e-06 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623064+0000", 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0000000000000002e-06 2024-09-16T22:17:36.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196403 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631253+0000", 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017878516000000001, 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024177596999999999, 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196403 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631253+0000", 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631253+0000", 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631254+0000", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631256+0000", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631263+0000", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631268+0000", 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196538 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641202+0000", 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079290739999999995, 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01428025, 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196538 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641202+0000", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641202+0000", 2024-09-16T22:17:36.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641203+0000", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641205+0000", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.889e-06 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641211+0000", 2024-09-16T22:17:36.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641216+0000", 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0780000000000003e-06 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066874217, 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073275684999999993, 2024-09-16T22:17:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061549315, 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068093637999999998, 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:17:36.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195877 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594109+0000", 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055023048999999997, 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061617077999999999, 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195877 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594109+0000", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594109+0000", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594109+0000", 2024-09-16T22:17:36.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594115+0000", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1059999999999996e-06 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594122+0000", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5740000000000001e-06 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594127+0000", 2024-09-16T22:17:36.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2549999999999997e-06 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196009 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604560+0000", 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044571870999999999, 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051211741999999998, 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196009 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604560+0000", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604560+0000", 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604560+0000", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8400000000000004e-07 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604563+0000", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604572+0000", 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3149999999999998e-06 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604578+0000", 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.665e-06 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196142 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.611249+0000", 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037882445000000001, 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044574856000000003, 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196142 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611249+0000", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611249+0000", 2024-09-16T22:17:36.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611250+0000", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611252+0000", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611258+0000", 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:36.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611263+0000", 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8409999999999999e-06 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196273 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.623072+0000", 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026059260000000001, 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032779230999999999, 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196273 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623072+0000", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623072+0000", 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623073+0000", 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623075+0000", 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4090000000000001e-06 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623082+0000", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8050000000000001e-06 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623100+0000", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7473999999999999e-05 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196404 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631277+0000", 2024-09-16T22:17:36.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017854950000000001, 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024625900999999999, 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196404 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631277+0000", 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631277+0000", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631277+0000", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631279+0000", 2024-09-16T22:17:36.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0159999999999998e-06 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631285+0000", 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4659999999999996e-06 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631294+0000", 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3499999999999997e-06 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196539 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.641223+0000", 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079087350000000001, 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014735408, 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196539 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641223+0000", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641223+0000", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641223+0000", 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641225+0000", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.996e-06 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641232+0000", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5440000000000004e-06 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.641237+0000", 2024-09-16T22:17:36.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.04e-06 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3.c7d414f0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066844766999999999, 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073730405999999998, 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3.16c566f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.061529045999999997, 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068465227000000003, 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195878 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.594134+0000", 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.054997745000000001, 2024-09-16T22:17:36.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061987469000000003, 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195878 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594134+0000", 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594134+0000", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594134+0000", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594136+0000", 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0090000000000002e-06 2024-09-16T22:17:36.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594142+0000", 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8989999999999998e-06 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.594147+0000", 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5310000000000003e-06 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196010 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.604586+0000", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.044545508999999997, 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051582337999999998, 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196010 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604586+0000", 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604586+0000", 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604586+0000", 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604589+0000", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.339e-06 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604596+0000", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9720000000000003e-06 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.604601+0000", 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-09-16T22:17:36.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196143 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.611270+0000", 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.037861961, 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044945298000000002, 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196143 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611270+0000", 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611270+0000", 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611270+0000", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611272+0000", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0379999999999998e-06 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611278+0000", 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6389999999999997e-06 2024-09-16T22:17:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.611282+0000", 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5140000000000003e-06 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196274 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.623118+0000", 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026014038999999999, 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033167542000000001, 2024-09-16T22:17:36.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196274 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623118+0000", 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623118+0000", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623118+0000", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623121+0000", 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623129+0000", 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.259e-06 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.623135+0000", 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:196405 3.10 3.c7d414f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.631301+0000", 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.017830558999999999, 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025033725, 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 196405 2024-09-16T22:17:36.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631301+0000", 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631301+0000", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631301+0000", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631304+0000", 2024-09-16T22:17:36.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1270000000000001e-06 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631310+0000", 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4570000000000002e-06 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.631315+0000", 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.159 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T22:17:36.159 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 154 2024-09-16T22:17:36.159 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:36.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:36.161 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5618 writesame 0x752b492 thru 0x7532945 data_size 0x4d(0x74b4 bytes) 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5619 read 0x3f4eefb thru 0x3f5e721 (0xf827 bytes) 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5620 punch from 0x13f81a5 to 0x14066dc, (0xe537 bytes) 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5622 trunc from 0x7532946 to 0x853a003 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5625 punch from 0x5517989 to 0x551b0b0, (0x3727 bytes) 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5627 read 0x21b25e8 thru 0x21c0b55 (0xe56e bytes) 2024-09-16T22:17:36.164 INFO:teuthology.orchestra.run.smithi031.stdout:5630 punch from 0x3428b39 to 0x342d025, (0x44ec bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5631 read 0x2546dac thru 0x25567c3 (0xfa18 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5632 writesame 0x2a8a836 thru 0x2a9a2f5 data_size 0x88(0xfac0 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5634 writesame 0x5bc4670 thru 0x5bcfe8a data_size 0x1(0xb81b bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5635 write 0x1c62942 thru 0x1c6e4c4 (0xbb83 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5636 read 0x14d02e8 thru 0x14d9766 (0x947f bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5638 write 0x53bf0a5 thru 0x53c482b (0x5787 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5640 writesame 0xade6a03 thru 0xadeb96c data_size 0x26(0x4f6a bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5643 punch from 0x510b8ea to 0x511440e, (0x8b24 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5646 punch from 0x5ed5ff1 to 0x5ee46d5, (0xe6e4 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5648 read 0xa873d89 thru 0xa874801 (0xa79 bytes) 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.165 INFO:teuthology.orchestra.run.smithi031.stdout:5651 read 0xa0d02d2 thru 0xa0dceb6 (0xcbe5 bytes) 2024-09-16T22:17:36.166 INFO:teuthology.orchestra.run.smithi031.stdout:5652 write 0x7433fee thru 0x7441f57 (0xdf6a bytes) 2024-09-16T22:17:36.166 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:36.162+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:36.169 INFO:teuthology.orchestra.run.smithi031.stdout:5653 write 0xb354c7c thru 0xb359965 (0x4cea bytes) 2024-09-16T22:17:36.173 INFO:teuthology.orchestra.run.smithi031.stdout:5654 read 0x7509a76 thru 0x750fcc8 (0x6253 bytes) 2024-09-16T22:17:36.173 INFO:teuthology.orchestra.run.smithi031.stdout:5656 write 0x1d137b3 thru 0x1d15d6a (0x25b8 bytes) 2024-09-16T22:17:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:5657 trunc from 0xb359966 to 0x5dddb5 2024-09-16T22:17:36.261 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:36.262+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:36.329 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:36.330 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:36.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:36.390 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:36.389+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:36.459 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.463 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:36.465+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:36.480 INFO:teuthology.orchestra.run.smithi031.stdout:5662 write 0xd9b4742 thru 0xd9bf8d8 (0xb197 bytes) 2024-09-16T22:17:36.484 INFO:teuthology.orchestra.run.smithi031.stdout:5664 write 0x73064f thru 0x7396bf (0x9071 bytes) 2024-09-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi031.stdout:5665 read 0x74b879f thru 0x74bbe91 (0x36f3 bytes) 2024-09-16T22:17:36.490 INFO:teuthology.orchestra.run.smithi031.stdout:5666 trunc from 0xd9bf8d9 to 0x93d10c9 2024-09-16T22:17:36.501 INFO:teuthology.orchestra.run.smithi031.stdout:5668 write 0x8e1762a thru 0x8e220f4 (0xaacb bytes) 2024-09-16T22:17:36.506 INFO:teuthology.orchestra.run.smithi031.stdout:5669 write 0x4202eb5 thru 0x420e1a7 (0xb2f3 bytes) 2024-09-16T22:17:36.510 INFO:teuthology.orchestra.run.smithi031.stdout:5670 read 0x4003412 thru 0x4004ee7 (0x1ad6 bytes) 2024-09-16T22:17:36.510 INFO:teuthology.orchestra.run.smithi031.stdout:5672 write 0x77dbb18 thru 0x77e6a04 (0xaeed bytes) 2024-09-16T22:17:36.514 INFO:teuthology.orchestra.run.smithi031.stdout:5674 writesame 0x80bd3be thru 0x80c4b4c data_size 0x1(0x778f bytes) 2024-09-16T22:17:36.526 INFO:teuthology.orchestra.run.smithi031.stdout:5675 punch from 0x95b576 to 0x961e6e, (0x68f8 bytes) 2024-09-16T22:17:36.526 INFO:teuthology.orchestra.run.smithi031.stdout:5676 write 0x744c61d thru 0x7453542 (0x6f26 bytes) 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94725986200000001, 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:17:36.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94724012000000002, 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:17:36.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:17:36.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94721067000000003, 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:17:36.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94718504299999995, 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94716239000000002, 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:17:36.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:36.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94713941199999996, 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:36.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94711635900000002, 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94709463900000002, 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:17:36.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:36.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94707443499999999, 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94705584399999998, 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:36.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:17:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94703431400000004, 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.941915218, 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94189494900000004, 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:17:36.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:17:36.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94187281899999997, 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:36.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:17:36.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:17:36.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94185333800000004, 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:17:36.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94182899799999997, 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:17:36.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:17:36.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94180903199999999, 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94178762100000002, 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:36.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:17:36.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94176585999999995, 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:36.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:17:36.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:17:36.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.94174539199999996, 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:36.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:36.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:36.614 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:5677 trunc from 0x93d10c9 to 0xd3dbb0b 2024-09-16T22:17:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:5678 punch from 0x8c32108 to 0x8c3c8f3, (0xa7eb bytes) 2024-09-16T22:17:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:5679 writesame 0xa17f089 thru 0xa18d7fd data_size 0x1(0xe775 bytes) 2024-09-16T22:17:36.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:5680 trunc from 0xd3dbb0b to 0x3aaef5a 2024-09-16T22:17:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:5682 read 0x3500afd thru 0x350f6d4 (0xebd8 bytes) 2024-09-16T22:17:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:5683 write 0xdec6397 thru 0xdec8547 (0x21b1 bytes) 2024-09-16T22:17:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:5685 read 0x3fd88a8 thru 0x3fdef72 (0x66cb bytes) 2024-09-16T22:17:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:5686 writesame 0x8f2a758 thru 0x8f2ad35 data_size 0x1(0x5de bytes) 2024-09-16T22:17:36.617 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:36.617+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:5691 write 0xb8b0017 thru 0xb8bf004 (0xefee bytes) 2024-09-16T22:17:36.619 INFO:teuthology.orchestra.run.smithi031.stdout:5692 read 0x535485e thru 0x5358ae0 (0x4283 bytes) 2024-09-16T22:17:36.619 INFO:teuthology.orchestra.run.smithi031.stdout:5693 writesame 0xadc5af1 thru 0xadcd9c2 data_size 0x15(0x7ed2 bytes) 2024-09-16T22:17:36.621 INFO:teuthology.orchestra.run.smithi031.stdout:5695 read 0xc9cb2a0 thru 0xc9d7890 (0xc5f1 bytes) 2024-09-16T22:17:36.621 INFO:teuthology.orchestra.run.smithi031.stdout:5696 write 0x2c4ecf2 thru 0x2c56e16 (0x8125 bytes) 2024-09-16T22:17:36.623 INFO:teuthology.orchestra.run.smithi031.stdout:5697 write 0xd1e5c1d thru 0xd1f5173 (0xf557 bytes) 2024-09-16T22:17:36.627 INFO:teuthology.orchestra.run.smithi031.stdout:5698 writesame 0xd544920 thru 0xd5524de data_size 0x1(0xdbbf bytes) 2024-09-16T22:17:36.655 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.655 INFO:teuthology.orchestra.run.smithi031.stdout:5700 read 0x7fc1e55 thru 0x7fcf3b9 (0xd565 bytes) 2024-09-16T22:17:36.655 INFO:teuthology.orchestra.run.smithi031.stdout:5701 read 0x8456baa thru 0x8459d2a (0x3181 bytes) 2024-09-16T22:17:36.655 INFO:teuthology.orchestra.run.smithi031.stdout:5702 write 0x56c418b thru 0x56d1e04 (0xdc7a bytes) 2024-09-16T22:17:36.659 INFO:teuthology.orchestra.run.smithi031.stdout:5703 read 0xb437e1a thru 0xb43fae1 (0x7cc8 bytes) 2024-09-16T22:17:36.660 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:36.660 INFO:teuthology.orchestra.run.smithi031.stdout:5706 write 0x6f2df91 thru 0x6f3cddf (0xee4f bytes) 2024-09-16T22:17:36.664 INFO:teuthology.orchestra.run.smithi031.stdout:5707 writesame 0x7d18943 thru 0x7d20d7e data_size 0x54(0x843c bytes) 2024-09-16T22:17:36.668 INFO:teuthology.orchestra.run.smithi031.stdout:5709 read 0x76abe7f thru 0x76b3b5b (0x7cdd bytes) 2024-09-16T22:17:36.668 INFO:teuthology.orchestra.run.smithi031.stdout:5710 read 0xbebd237 thru 0xbeca5e4 (0xd3ae bytes) 2024-09-16T22:17:36.668 INFO:teuthology.orchestra.run.smithi031.stdout:5711 writesame 0x25113 thru 0x2ce95 data_size 0x17(0x7d83 bytes) 2024-09-16T22:17:36.671 INFO:teuthology.orchestra.run.smithi031.stdout:5712 write 0x5cc95b7 thru 0x5cd26fd (0x9147 bytes) 2024-09-16T22:17:36.672 INFO:teuthology.orchestra.run.smithi031.stdout:5713 write 0x7c7ae87 thru 0x7c7b7f1 (0x96b bytes) 2024-09-16T22:17:36.675 INFO:teuthology.orchestra.run.smithi031.stdout:5714 write 0xb47852e thru 0xb480bde (0x86b1 bytes) 2024-09-16T22:17:36.679 INFO:teuthology.orchestra.run.smithi031.stdout:5715 read 0x95ad73d thru 0x95b6c09 (0x94cd bytes) 2024-09-16T22:17:36.679 INFO:teuthology.orchestra.run.smithi031.stdout:5716 punch from 0x644d89f to 0x64504c9, (0x2c2a bytes) 2024-09-16T22:17:36.680 INFO:teuthology.orchestra.run.smithi031.stdout:5717 read 0x55ef598 thru 0x55ff0d4 (0xfb3d bytes) 2024-09-16T22:17:36.680 INFO:teuthology.orchestra.run.smithi031.stdout:5719 writesame 0xbc0b6c8 thru 0xbc1b377 data_size 0x68(0xfcb0 bytes) 2024-09-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi031.stdout:5720 write 0xb28db5a thru 0xb2909a3 (0x2e4a bytes) 2024-09-16T22:17:36.684 INFO:teuthology.orchestra.run.smithi031.stdout:5722 write 0x8ea07e thru 0x8f446f (0xa3f2 bytes) 2024-09-16T22:17:36.686 INFO:teuthology.orchestra.run.smithi031.stdout:5723 read 0xbe7b065 thru 0xbe86a61 (0xb9fd bytes) 2024-09-16T22:17:36.686 INFO:teuthology.orchestra.run.smithi031.stdout:5724 write 0x943dab8 thru 0x9447c2a (0xa173 bytes) 2024-09-16T22:17:36.690 INFO:teuthology.orchestra.run.smithi031.stdout:5725 write 0x466920b thru 0x46791f7 (0xffed bytes) 2024-09-16T22:17:36.694 INFO:teuthology.orchestra.run.smithi031.stdout:5726 read 0xbe85f0 thru 0xbf1c97 (0x96a8 bytes) 2024-09-16T22:17:36.694 INFO:teuthology.orchestra.run.smithi031.stdout:5727 write 0x9681d1b thru 0x9684dd3 (0x30b9 bytes) 2024-09-16T22:17:36.695 INFO:teuthology.orchestra.run.smithi031.stdout:5728 write 0x25b2aeb thru 0x25b9920 (0x6e36 bytes) 2024-09-16T22:17:36.697 INFO:teuthology.orchestra.run.smithi031.stdout:5729 write 0xcba821c thru 0xcbb3e42 (0xbc27 bytes) 2024-09-16T22:17:36.699 INFO:teuthology.orchestra.run.smithi031.stdout:5730 read 0xd7868e8 thru 0xd79677f (0xfe98 bytes) 2024-09-16T22:17:36.699 INFO:teuthology.orchestra.run.smithi031.stdout:5731 write 0x1a6a6a4 thru 0x1a6a6ec (0x49 bytes) 2024-09-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi031.stdout:5732 clone 46 order 23 su 262144 sc 4 2024-09-16T22:17:36.716 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:36.718+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:36.735 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:36.735 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:36.735 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:36.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:36.827 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:36.829+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:36.891 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:36.891 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:36.891 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:36.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:36.918 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:36.920+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:37.011 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:37.011 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:37.011 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T22:17:37.011 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:37.011 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:37.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T22:17:37.018 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:37.020+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:37.048 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.472682737, 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:37.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.472663396999998, 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:37.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:37.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.472258996999997, 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:37.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.470563726999998, 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:37.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.470533193999998, 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.466132295000001, 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:37.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.466111814000001, 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:37.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.466091237999997, 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.466066753, 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.466045182000002, 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459966584, 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:37.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:37.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:37.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459946252000002, 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:37.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459925032999998, 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:37.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459903660999998, 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459882823000001, 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459862242, 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.459840675000002, 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:37.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:37.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.455193801, 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:37.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:37.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3998884, 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:17:37.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:17:37.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:37.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.72269941699999996, 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:17:37.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.138 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:37.140 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T22:17:37.140 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:37.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:37.141 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:37.141+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:37.240 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:37.242+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:37.288 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:37.289 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:37.289 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:37.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:37.341 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:37.343+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:37.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:37.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone45 from 0xdec8548 (overlap 0x5dddb5) to 0x962f8 2024-09-16T22:17:37.442 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:37.444+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:37.525 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-16T22:17:37.526 INFO:teuthology.orchestra.run.smithi031.stdout:5733 punch from 0x5accca6 to 0x5ada3d4, (0xd72e bytes) 2024-09-16T22:17:37.530 INFO:teuthology.orchestra.run.smithi031.stdout:5734 write 0x55e46fa thru 0x55ea9c8 (0x62cf bytes) 2024-09-16T22:17:37.537 INFO:teuthology.orchestra.run.smithi031.stdout:5735 writesame 0x7cef438 thru 0x7cfd3db data_size 0xd(0xdfa4 bytes) 2024-09-16T22:17:37.543 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:37.545+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:37.543 INFO:teuthology.orchestra.run.smithi031.stdout:5736 write 0x8dfe141 thru 0x8e0d5e0 (0xf4a0 bytes) 2024-09-16T22:17:37.550 INFO:teuthology.orchestra.run.smithi031.stdout:5740 write 0xd063d04 thru 0xd072be2 (0xeedf bytes) 2024-09-16T22:17:37.558 INFO:teuthology.orchestra.run.smithi031.stdout:5742 write 0xb8b6262 thru 0xb8bf58c (0x932b bytes) 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.133852434000005, 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.299047381999998, 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.299026542999997, 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.299003302999999, 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.298301418000001, 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:37.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.295168971000003, 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:37.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:37.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.294759360999997, 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:37.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.294473183999997, 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:37.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.294175959999997, 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:37.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.293387717000002, 2024-09-16T22:17:37.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:37.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.293272948000002, 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:37.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:37.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.293021731000003, 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.292713372999998, 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:37.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.292674435999999, 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:37.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.292389016000001, 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.286537512999999, 2024-09-16T22:17:37.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:37.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:37.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.286513114999998, 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:37.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.286462465, 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:37.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:37.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.286428768, 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:37.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:37.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.286317797999999, 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:37.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:37.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:37.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:37.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:37.651 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:37.652 INFO:teuthology.orchestra.run.smithi031.stdout:5743 write 0x8f47332 thru 0x8f4f482 (0x8151 bytes) 2024-09-16T22:17:37.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:5744 trunc from 0xdec8548 to 0xf4ec9b 2024-09-16T22:17:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:5745 trunc from 0xf4ec9b to 0x4ac19f9 2024-09-16T22:17:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:5747 trunc from 0x4ac19f9 to 0x3940d29 2024-09-16T22:17:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:5748 write 0x643e3dd thru 0x644d13a (0xed5e bytes) 2024-09-16T22:17:37.654 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:37.654+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:37.656 INFO:teuthology.orchestra.run.smithi031.stdout:5750 read 0x3a6aaff thru 0x3a6c095 (0x1597 bytes) 2024-09-16T22:17:37.668 INFO:teuthology.orchestra.run.smithi031.stdout:5751 writesame 0xc7a154a thru 0xc7ac050 data_size 0x1(0xab07 bytes) 2024-09-16T22:17:37.669 INFO:teuthology.orchestra.run.smithi031.stdout:5752 read 0x91eada3 thru 0x91ee741 (0x399f bytes) 2024-09-16T22:17:37.669 INFO:teuthology.orchestra.run.smithi031.stdout:5758 write 0x7ce2d2c thru 0x7ce98c9 (0x6b9e bytes) 2024-09-16T22:17:37.672 INFO:teuthology.orchestra.run.smithi031.stdout:5759 read 0x34e7fb3 thru 0x34eec97 (0x6ce5 bytes) 2024-09-16T22:17:37.674 INFO:teuthology.orchestra.run.smithi031.stdout:5760 trunc from 0xc7ac051 to 0xedf5eae 2024-09-16T22:17:37.679 INFO:teuthology.orchestra.run.smithi031.stdout:5761 read 0x1bb1a47 thru 0x1bbc7c0 (0xad7a bytes) 2024-09-16T22:17:37.695 INFO:teuthology.orchestra.run.smithi031.stdout:5765 trunc from 0xedf5eae to 0x12e9b85 2024-09-16T22:17:37.718 INFO:teuthology.orchestra.run.smithi031.stdout:5766 write 0x4f737e4 thru 0x4f7c6dd (0x8efa bytes) 2024-09-16T22:17:37.723 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.730 INFO:teuthology.orchestra.run.smithi031.stdout:5770 trunc from 0x4f7c6de to 0xba096de 2024-09-16T22:17:37.736 INFO:teuthology.orchestra.run.smithi031.stdout:5771 punch from 0x3b43fb8 to 0x3b4e144, (0xa18c bytes) 2024-09-16T22:17:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:5772 writesame 0xcea593d thru 0xcea84c7 data_size 0x1(0x2b8b bytes) 2024-09-16T22:17:37.742 INFO:teuthology.orchestra.run.smithi031.stdout:5774 writesame 0x9b9b62a thru 0x9b9cb12 data_size 0x1(0x14e9 bytes) 2024-09-16T22:17:37.747 INFO:teuthology.orchestra.run.smithi031.stdout:5775 write 0x1896350 thru 0x189c254 (0x5f05 bytes) 2024-09-16T22:17:37.748 INFO:teuthology.orchestra.run.smithi031.stdout:5776 read 0x23d02e1 thru 0x23db0e2 (0xae02 bytes) 2024-09-16T22:17:37.753 INFO:teuthology.orchestra.run.smithi031.stdout:5777 trunc from 0xcea84c8 to 0x8af7413 2024-09-16T22:17:37.753 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:37.755+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:37.764 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:37.764 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:37.770 INFO:teuthology.orchestra.run.smithi031.stdout:5779 write 0xddef98e thru 0xddfc6cf (0xcd42 bytes) 2024-09-16T22:17:37.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:37.781 INFO:teuthology.orchestra.run.smithi031.stdout:5781 trunc from 0xddfc6d0 to 0x7615a90 2024-09-16T22:17:37.791 INFO:teuthology.orchestra.run.smithi031.stdout:5782 write 0x3ae4de3 thru 0x3ae60c9 (0x12e7 bytes) 2024-09-16T22:17:37.809 INFO:teuthology.orchestra.run.smithi031.stdout:5783 writesame 0xe71dda7 thru 0xe71fc3c data_size 0x3a(0x1e96 bytes) 2024-09-16T22:17:37.809 INFO:teuthology.orchestra.run.smithi031.stdout:5784 write 0x5413d2 thru 0x549741 (0x8370 bytes) 2024-09-16T22:17:37.811 INFO:teuthology.orchestra.run.smithi031.stdout:5785 write 0xabd90ea thru 0xabe258a (0x94a1 bytes) 2024-09-16T22:17:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:5786 trunc from 0xe71fc3d to 0x1950aee 2024-09-16T22:17:37.846 INFO:teuthology.orchestra.run.smithi031.stdout:5788 write 0x1c21520 thru 0x1c28c2e (0x770f bytes) 2024-09-16T22:17:37.848 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.848 INFO:teuthology.orchestra.run.smithi031.stdout:5790 punch from 0x13f80f7 to 0x1407a98, (0xf9a1 bytes) 2024-09-16T22:17:37.858 INFO:teuthology.orchestra.run.smithi031.stdout:5791 write 0x6985632 thru 0x6985670 (0x3f bytes) 2024-09-16T22:17:37.859 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:37.861+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:37.861 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.861 INFO:teuthology.orchestra.run.smithi031.stdout:5793 writesame 0x5f76d92 thru 0x5f7a114 data_size 0x1(0x3383 bytes) 2024-09-16T22:17:37.877 INFO:teuthology.orchestra.run.smithi031.stdout:5794 trunc from 0x6985671 to 0x488b93 2024-09-16T22:17:37.896 INFO:teuthology.orchestra.run.smithi031.stdout:5795 writesame 0x101c191 thru 0x101c815 data_size 0x1(0x685 bytes) 2024-09-16T22:17:37.898 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.898 INFO:teuthology.orchestra.run.smithi031.stdout:5798 punch from 0x338dc0 to 0x347bf5, (0xee35 bytes) 2024-09-16T22:17:37.898 INFO:teuthology.orchestra.run.smithi031.stdout:5799 read 0x23fdb2 thru 0x24bb3a (0xbd89 bytes) 2024-09-16T22:17:37.904 INFO:teuthology.orchestra.run.smithi031.stdout:5801 trunc from 0x101c816 to 0x5fe1234 2024-09-16T22:17:37.909 INFO:teuthology.orchestra.run.smithi031.stdout:5803 writesame 0x5b5b30d thru 0x5b6458f data_size 0x1(0x9283 bytes) 2024-09-16T22:17:37.924 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:37.924 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:37.932 INFO:teuthology.orchestra.run.smithi031.stdout:5804 trunc from 0x5fe1234 to 0x31a3951 2024-09-16T22:17:37.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:37.941 INFO:teuthology.orchestra.run.smithi031.stdout:5805 read 0x18512bc thru 0x1857ce1 (0x6a26 bytes) 2024-09-16T22:17:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:5809 punch from 0x1324c81 to 0x1329ca8, (0x5027 bytes) 2024-09-16T22:17:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:5812 read 0x106b95d thru 0x1070579 (0x4c1d bytes) 2024-09-16T22:17:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:5814 read 0x1ba354b thru 0x1badbe4 (0xa69a bytes) 2024-09-16T22:17:37.948 INFO:teuthology.orchestra.run.smithi031.stdout:5815 write 0x346249d thru 0x346938b (0x6eef bytes) 2024-09-16T22:17:37.951 INFO:teuthology.orchestra.run.smithi031.stdout:5816 write 0x6628a5 thru 0x668a37 (0x6193 bytes) 2024-09-16T22:17:37.952 INFO:teuthology.orchestra.run.smithi031.stdout:5817 write 0x1fca61c thru 0x1fd488a (0xa26f bytes) 2024-09-16T22:17:37.954 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:37.956+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:37.968 INFO:teuthology.orchestra.run.smithi031.stdout:5818 write 0xaceaa31 thru 0xacf8efa (0xe4ca bytes) 2024-09-16T22:17:37.971 INFO:teuthology.orchestra.run.smithi031.stdout:5821 writesame 0xa64ce88 thru 0xa64e02b data_size 0x2(0x11a4 bytes) 2024-09-16T22:17:37.974 INFO:teuthology.orchestra.run.smithi031.stdout:5822 punch from 0x3e6a413 to 0x3e6c4ce, (0x20bb bytes) 2024-09-16T22:17:37.974 INFO:teuthology.orchestra.run.smithi031.stdout:5824 read 0x86c290c thru 0x86d1811 (0xef06 bytes) 2024-09-16T22:17:37.975 INFO:teuthology.orchestra.run.smithi031.stdout:5825 punch from 0xfc3db2 to 0xfc3dc9, (0x17 bytes) 2024-09-16T22:17:37.975 INFO:teuthology.orchestra.run.smithi031.stdout:5826 trunc from 0xacf8efb to 0x9be2455 2024-09-16T22:17:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:5828 read 0x9a4cc9d thru 0x9a58407 (0xb76b bytes) 2024-09-16T22:17:37.991 INFO:teuthology.orchestra.run.smithi031.stdout:5829 trunc from 0x9be2455 to 0x4536894 2024-09-16T22:17:37.996 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:37.996 INFO:teuthology.orchestra.run.smithi031.stdout:5832 punch from 0xa2d7a7 to 0xa3bd17, (0xe570 bytes) 2024-09-16T22:17:37.996 INFO:teuthology.orchestra.run.smithi031.stdout:5833 read 0x416e086 thru 0x416f7bc (0x1737 bytes) 2024-09-16T22:17:37.997 INFO:teuthology.orchestra.run.smithi031.stdout:5834 write 0x173b2e1 thru 0x1741105 (0x5e25 bytes) 2024-09-16T22:17:38.009 INFO:teuthology.orchestra.run.smithi031.stdout:5835 writesame 0x8ca6279 thru 0x8caae3b data_size 0xf(0x4bc3 bytes) 2024-09-16T22:17:38.009 INFO:teuthology.orchestra.run.smithi031.stdout:5836 read 0x7855665 thru 0x785fdfb (0xa797 bytes) 2024-09-16T22:17:38.009 INFO:teuthology.orchestra.run.smithi031.stdout:5837 punch from 0x70e9458 to 0x70ef28a, (0x5e32 bytes) 2024-09-16T22:17:38.009 INFO:teuthology.orchestra.run.smithi031.stdout:5838 writesame 0x45c7ff4 thru 0x45cf2e9 data_size 0x1e(0x72f6 bytes) 2024-09-16T22:17:38.012 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.013 INFO:teuthology.orchestra.run.smithi031.stdout:5841 read 0x7a13a52 thru 0x7a14b86 (0x1135 bytes) 2024-09-16T22:17:38.013 INFO:teuthology.orchestra.run.smithi031.stdout:5842 read 0x51d45ae thru 0x51d6b24 (0x2577 bytes) 2024-09-16T22:17:38.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.017 INFO:teuthology.orchestra.run.smithi031.stdout:5845 trunc from 0x8caae3c to 0xd0bf453 2024-09-16T22:17:38.024 INFO:teuthology.orchestra.run.smithi031.stdout:5847 punch from 0x2b9447f to 0x2b96afa, (0x267b bytes) 2024-09-16T22:17:38.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.024 INFO:teuthology.orchestra.run.smithi031.stdout:5851 writesame 0x8e063ff thru 0x8e0e0d8 data_size 0x15(0x7cda bytes) 2024-09-16T22:17:38.027 INFO:teuthology.orchestra.run.smithi031.stdout:5852 read 0x1a6d816 thru 0x1a72079 (0x4864 bytes) 2024-09-16T22:17:38.029 INFO:teuthology.orchestra.run.smithi031.stdout:5853 trunc from 0xd0bf453 to 0xe1ab602 2024-09-16T22:17:38.034 INFO:teuthology.orchestra.run.smithi031.stdout:5854 writesame 0xabccc1e thru 0xabccfb2 data_size 0x1(0x395 bytes) 2024-09-16T22:17:38.037 INFO:teuthology.orchestra.run.smithi031.stdout:5855 write 0xaa2a7af thru 0xaa35c21 (0xb473 bytes) 2024-09-16T22:17:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:5856 punch from 0x3ec0191 to 0x3ecf00b, (0xee7a bytes) 2024-09-16T22:17:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:5857 write 0x3b026fa thru 0x3b0a1e4 (0x7aeb bytes) 2024-09-16T22:17:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:5858 flatten 2024-09-16T22:17:38.051 INFO:teuthology.orchestra.run.smithi031.stdout:5859 punch from 0x4438d55 to 0x443a3de, (0x1689 bytes) 2024-09-16T22:17:38.051 INFO:teuthology.orchestra.run.smithi031.stdout:5860 write 0x16cb532 thru 0x16ccccc (0x179b bytes) 2024-09-16T22:17:38.052 INFO:teuthology.orchestra.run.smithi031.stdout:5861 write 0x8a42895 thru 0x8a49b6b (0x72d7 bytes) 2024-09-16T22:17:38.055 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:38.057+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:38.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.056 INFO:teuthology.orchestra.run.smithi031.stdout:5863 writesame 0x90c5f29 thru 0x90c6391 data_size 0x1(0x469 bytes) 2024-09-16T22:17:38.059 INFO:teuthology.orchestra.run.smithi031.stdout:5864 writesame 0xab8cb26 thru 0xab9972b data_size 0xf(0xcc06 bytes) 2024-09-16T22:17:38.063 INFO:teuthology.orchestra.run.smithi031.stdout:5866 punch from 0x95ff286 to 0x9607bea, (0x8964 bytes) 2024-09-16T22:17:38.076 INFO:teuthology.orchestra.run.smithi031.stdout:5867 trunc from 0xe1ab602 to 0x2a800ba 2024-09-16T22:17:38.083 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493715674000001, 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493677099999999, 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493657451000001, 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493616037999999, 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493580187999999, 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493562265000001, 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493542684000001, 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493517173000001, 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493498791, 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493480251000001, 2024-09-16T22:17:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493462071, 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493444504999999, 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493422543000001, 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493404619, 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493387109, 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.493369437999998, 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.492927497, 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:38.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.492812199999999, 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:38.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:38.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.492785201, 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:38.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.492731824, 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:38.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:38.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:38.163 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:38.163 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:38.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:38.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:38.165 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:38.165+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5868 trunc from 0x2a800ba to 0x9d96df1 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5870 trunc from 0x9d96df1 to 0x136f8e6 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5871 trunc from 0x136f8e6 to 0x7f4cf5c 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5872 write 0x21cf056 thru 0x21d049d (0x1448 bytes) 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5874 write 0xb05ac6b thru 0xb05d617 (0x29ad bytes) 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5877 read 0x1126d96 thru 0x1129ab1 (0x2d1c bytes) 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5879 write 0x2041163 thru 0x2041231 (0xcf bytes) 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5881 writesame 0x838269c thru 0x8391470 data_size 0x63(0xedd5 bytes) 2024-09-16T22:17:38.166 INFO:teuthology.orchestra.run.smithi031.stdout:5882 write 0x158707c thru 0x15938b6 (0xc83b bytes) 2024-09-16T22:17:38.168 INFO:teuthology.orchestra.run.smithi031.stdout:5884 write 0xc080f58 thru 0xc08fee9 (0xef92 bytes) 2024-09-16T22:17:38.178 INFO:teuthology.orchestra.run.smithi031.stdout:5885 writesame 0xddfcf24 thru 0xde0aff6 data_size 0xf(0xe0d3 bytes) 2024-09-16T22:17:38.178 INFO:teuthology.orchestra.run.smithi031.stdout:5886 write 0x322ca0a thru 0x3236639 (0x9c30 bytes) 2024-09-16T22:17:38.182 INFO:teuthology.orchestra.run.smithi031.stdout:5888 read 0xa0e4be3 thru 0xa0f0ccd (0xc0eb bytes) 2024-09-16T22:17:38.182 INFO:teuthology.orchestra.run.smithi031.stdout:5889 read 0x9583ec5 thru 0x9587efe (0x403a bytes) 2024-09-16T22:17:38.182 INFO:teuthology.orchestra.run.smithi031.stdout:5890 read 0x58ccaf5 thru 0x58d0401 (0x390d bytes) 2024-09-16T22:17:38.182 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.183 INFO:teuthology.orchestra.run.smithi031.stdout:5892 read 0x698777c thru 0x699725f (0xfae4 bytes) 2024-09-16T22:17:38.183 INFO:teuthology.orchestra.run.smithi031.stdout:5893 write 0x24f845e thru 0x24fbc19 (0x37bc bytes) 2024-09-16T22:17:38.184 INFO:teuthology.orchestra.run.smithi031.stdout:5894 read 0x20c6143 thru 0x20d12df (0xb19d bytes) 2024-09-16T22:17:38.184 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.184 INFO:teuthology.orchestra.run.smithi031.stdout:5898 write 0x11a3aa7 thru 0x11afe9b (0xc3f5 bytes) 2024-09-16T22:17:38.186 INFO:teuthology.orchestra.run.smithi031.stdout:5899 read 0xb75892f thru 0xb767829 (0xeefb bytes) 2024-09-16T22:17:38.186 INFO:teuthology.orchestra.run.smithi031.stdout:5900 write 0x7504a22 thru 0x7505add (0x10bc bytes) 2024-09-16T22:17:38.189 INFO:teuthology.orchestra.run.smithi031.stdout:5902 read 0xb3ca076 thru 0xb3cca66 (0x29f1 bytes) 2024-09-16T22:17:38.189 INFO:teuthology.orchestra.run.smithi031.stdout:5903 punch from 0x6b8c59 to 0x6c8151, (0xf4f8 bytes) 2024-09-16T22:17:38.189 INFO:teuthology.orchestra.run.smithi031.stdout:5905 write 0xd0de11 thru 0xd13c8d (0x5e7d bytes) 2024-09-16T22:17:38.192 INFO:teuthology.orchestra.run.smithi031.stdout:5906 write 0x4e27dcb thru 0x4e37aae (0xfce4 bytes) 2024-09-16T22:17:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:5907 write 0x674659f thru 0x6749047 (0x2aa9 bytes) 2024-09-16T22:17:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:5909 read 0x113a50b thru 0x113d563 (0x3059 bytes) 2024-09-16T22:17:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:5910 read 0x1161eb7 thru 0x1167ab1 (0x5bfb bytes) 2024-09-16T22:17:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:5912 read 0x3915121 thru 0x39153ea (0x2ca bytes) 2024-09-16T22:17:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:5914 write 0x4f9acfc thru 0x4fa0ec7 (0x61cc bytes) 2024-09-16T22:17:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:5917 writesame 0x9b4aef6 thru 0x9b4ea9b data_size 0xf(0x3ba6 bytes) 2024-09-16T22:17:38.209 INFO:teuthology.orchestra.run.smithi031.stdout:5918 write 0x8059965 thru 0x8061933 (0x7fcf bytes) 2024-09-16T22:17:38.209 INFO:teuthology.orchestra.run.smithi031.stdout:5919 read 0xdaf4dbf thru 0xdafbc47 (0x6e89 bytes) 2024-09-16T22:17:38.209 INFO:teuthology.orchestra.run.smithi031.stdout:5920 write 0x8705567 thru 0x8706a42 (0x14dc bytes) 2024-09-16T22:17:38.210 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.210 INFO:teuthology.orchestra.run.smithi031.stdout:5923 punch from 0x89f4242 to 0x89f6e54, (0x2c12 bytes) 2024-09-16T22:17:38.215 INFO:teuthology.orchestra.run.smithi031.stdout:5924 trunc from 0xde0aff7 to 0x9c80883 2024-09-16T22:17:38.227 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.227 INFO:teuthology.orchestra.run.smithi031.stdout:5926 read 0x8432791 thru 0x843516f (0x29df bytes) 2024-09-16T22:17:38.228 INFO:teuthology.orchestra.run.smithi031.stdout:5927 write 0x6a1aeeb thru 0x6a2a14f (0xf265 bytes) 2024-09-16T22:17:38.229 INFO:teuthology.orchestra.run.smithi031.stdout:5928 write 0x25ffaa1 thru 0x260fa24 (0xff84 bytes) 2024-09-16T22:17:38.231 INFO:teuthology.orchestra.run.smithi031.stdout:5929 punch from 0x29097b8 to 0x29150ee, (0xb936 bytes) 2024-09-16T22:17:38.234 INFO:teuthology.orchestra.run.smithi031.stdout:5931 trunc from 0x9c80883 to 0x719cc3b 2024-09-16T22:17:38.256 INFO:teuthology.orchestra.run.smithi031.stdout:5932 writesame 0x918c425 thru 0x919a279 data_size 0x2f(0xde55 bytes) 2024-09-16T22:17:38.256 INFO:teuthology.orchestra.run.smithi031.stdout:5934 read 0x65b278b thru 0x65ba0fe (0x7974 bytes) 2024-09-16T22:17:38.259 INFO:teuthology.orchestra.run.smithi031.stdout:5936 trunc from 0x919a27a to 0xbc5646c 2024-09-16T22:17:38.264 INFO:teuthology.orchestra.run.smithi031.stdout:5937 punch from 0x17e0950 to 0x17f04fa, (0xfbaa bytes) 2024-09-16T22:17:38.265 INFO:teuthology.orchestra.run.smithi031.stdout:5938 read 0x558909a thru 0x558b8c7 (0x282e bytes) 2024-09-16T22:17:38.265 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:38.266+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:38.265 INFO:teuthology.orchestra.run.smithi031.stdout:5939 read 0x403bd57 thru 0x4042ed3 (0x717d bytes) 2024-09-16T22:17:38.267 INFO:teuthology.orchestra.run.smithi031.stdout:5941 trunc from 0xbc5646c to 0xda6d2fa 2024-09-16T22:17:38.273 INFO:teuthology.orchestra.run.smithi031.stdout:5942 write 0x30c28fd thru 0x30d135d (0xea61 bytes) 2024-09-16T22:17:38.275 INFO:teuthology.orchestra.run.smithi031.stdout:5943 read 0xd8f0759 thru 0xd8fc530 (0xbdd8 bytes) 2024-09-16T22:17:38.275 INFO:teuthology.orchestra.run.smithi031.stdout:5945 punch from 0x1b69079 to 0x1b72a83, (0x9a0a bytes) 2024-09-16T22:17:38.275 INFO:teuthology.orchestra.run.smithi031.stdout:5949 read 0x9b9f984 thru 0x9baa1ee (0xa86b bytes) 2024-09-16T22:17:38.284 INFO:teuthology.orchestra.run.smithi031.stdout:5950 trunc from 0xda6d2fa to 0x55cdce5 2024-09-16T22:17:38.296 INFO:teuthology.orchestra.run.smithi031.stdout:5952 read 0x394f9ff thru 0x395bffd (0xc5ff bytes) 2024-09-16T22:17:38.296 INFO:teuthology.orchestra.run.smithi031.stdout:5953 read 0x464d7f6 thru 0x4654196 (0x69a1 bytes) 2024-09-16T22:17:38.303 INFO:teuthology.orchestra.run.smithi031.stdout:5954 writesame 0x5e904ab thru 0x5e97044 data_size 0x3(0x6b9a bytes) 2024-09-16T22:17:38.304 INFO:teuthology.orchestra.run.smithi031.stdout:5955 writesame 0x11174af thru 0x1124a4d data_size 0x1(0xd59f bytes) 2024-09-16T22:17:38.305 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:38.305 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:38.319 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:38.319 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:38.319 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:38.319 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:38.322 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:38.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:38.334 INFO:teuthology.orchestra.run.smithi031.stdout:5956 write 0x172f304 thru 0x173d5f4 (0xe2f1 bytes) 2024-09-16T22:17:38.335 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.336 INFO:teuthology.orchestra.run.smithi031.stdout:5958 write 0x1e6690b thru 0x1e762e4 (0xf9da bytes) 2024-09-16T22:17:38.339 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:38.348 INFO:teuthology.orchestra.run.smithi031.stdout:5959 writesame 0xa7c68fb thru 0xa7c778c data_size 0x5(0xe92 bytes) 2024-09-16T22:17:38.355 INFO:teuthology.orchestra.run.smithi031.stdout:5960 trunc from 0xa7c778d to 0x58b5c4d 2024-09-16T22:17:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:17:38.363 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:38.366 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:38.368+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:38.374 INFO:teuthology.orchestra.run.smithi031.stdout:5961 trunc from 0x58b5c4d to 0x74f102 2024-09-16T22:17:38.384 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:38.384 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:38.387 INFO:teuthology.orchestra.run.smithi031.stdout:5962 read 0x23c7d thru 0x294ec (0x5870 bytes) 2024-09-16T22:17:38.401 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:38.401 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:38.406 INFO:teuthology.orchestra.run.smithi031.stdout:5963 write 0xaea38e5 thru 0xaea96f0 (0x5e0c bytes) 2024-09-16T22:17:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.417 INFO:teuthology.orchestra.run.smithi031.stdout:5965 writesame 0xcc22674 thru 0xcc2daaa data_size 0x1(0xb437 bytes) 2024-09-16T22:17:38.419 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:38.425 INFO:teuthology.orchestra.run.smithi031.stdout:5969 writesame 0xe3deedd thru 0xe3e935c data_size 0x8(0xa480 bytes) 2024-09-16T22:17:38.425 INFO:teuthology.orchestra.run.smithi031.stdout:5970 writesame 0xddc17c1 thru 0xddce9ec data_size 0x2(0xd22c bytes) 2024-09-16T22:17:38.437 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:38.439 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:38.439 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:38.441 INFO:teuthology.orchestra.run.smithi031.stdout:5971 punch from 0xe2adc42 to 0xe2b4c66, (0x7024 bytes) 2024-09-16T22:17:38.441 INFO:teuthology.orchestra.run.smithi031.stdout:5972 read 0x867ef25 thru 0x868bba3 (0xcc7f bytes) 2024-09-16T22:17:38.441 INFO:teuthology.orchestra.run.smithi031.stdout:5975 read 0x5a49378 thru 0x5a520c7 (0x8d50 bytes) 2024-09-16T22:17:38.441 INFO:teuthology.orchestra.run.smithi031.stdout:5976 write 0x999cdad thru 0x99a21fa (0x544e bytes) 2024-09-16T22:17:38.445 INFO:teuthology.orchestra.run.smithi031.stdout:5977 punch from 0x4488ba7 to 0x4495e57, (0xd2b0 bytes) 2024-09-16T22:17:38.445 INFO:teuthology.orchestra.run.smithi031.stdout:5979 writesame 0x943b2ea thru 0x944624a data_size 0x13(0xaf61 bytes) 2024-09-16T22:17:38.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:38.450 INFO:teuthology.orchestra.run.smithi031.stdout:5980 writesame 0x4b50b42 thru 0x4b5a243 data_size 0x22(0x9702 bytes) 2024-09-16T22:17:38.453 INFO:teuthology.orchestra.run.smithi031.stdout:5981 write 0xa3088ac thru 0xa3101f5 (0x794a bytes) 2024-09-16T22:17:38.456 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.462 INFO:teuthology.orchestra.run.smithi031.stdout:5984 write 0xec45c59 thru 0xec51874 (0xbc1c bytes) 2024-09-16T22:17:38.466 INFO:teuthology.orchestra.run.smithi031.stdout:5985 write 0x4316aa8 thru 0x431b367 (0x48c0 bytes) 2024-09-16T22:17:38.475 INFO:teuthology.orchestra.run.smithi031.stdout:5988 trunc from 0xec51875 to 0x912058c 2024-09-16T22:17:38.493 INFO:teuthology.orchestra.run.smithi031.stdout:5989 write 0x9bc0b22 thru 0x9bc2ba4 (0x2083 bytes) 2024-09-16T22:17:38.494 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:38.496+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:5991 trunc from 0x9bc2ba5 to 0xbc4fa3a 2024-09-16T22:17:38.504 INFO:teuthology.orchestra.run.smithi031.stdout:5992 read 0xc27063 thru 0xc2e52e (0x74cc bytes) 2024-09-16T22:17:38.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.504 INFO:teuthology.orchestra.run.smithi031.stdout:5995 read 0x9f14da8 thru 0x9f1e7f7 (0x9a50 bytes) 2024-09-16T22:17:38.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-16T22:17:38.515 INFO:teuthology.orchestra.run.smithi031.stdout:5997 trunc from 0xbc4fa3a to 0x27b2eee 2024-09-16T22:17:38.539 INFO:teuthology.orchestra.run.smithi031.stdout:5998 trunc from 0x27b2eee to 0xea75bee 2024-09-16T22:17:38.545 INFO:teuthology.orchestra.run.smithi031.stdout:5999 read 0x3afb2e3 thru 0x3b07f17 (0xcc35 bytes) 2024-09-16T22:17:38.545 INFO:teuthology.orchestra.run.smithi031.stdout:6000 write 0xc1cc39a thru 0xc1d1c1a (0x5881 bytes) 2024-09-16T22:17:38.558 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:38.558 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:38.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:38.568 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:38.569+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:38.578 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-16T22:17:38.669 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:38.671+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:38.715 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:38.715 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:38.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:38.770 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:38.772+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:38.826 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:38.826 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:38.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:38.878 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:38.880+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:38.937 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:38.937 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:38.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:38.972 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:38.974+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:39.073 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:39.075+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:39.089 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.089 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:39.174 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:39.176+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:39.241 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.241 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:39.275 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:39.277+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:39.353 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.353 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:39.375 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:39.377+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:39.477 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:39.478+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:39.500 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.500 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:39.578 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:39.580+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:39.656 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.657 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:39.679 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:39.681+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:39.771 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.771 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:39.781 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:39.782+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:39.881 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:39.882+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:39.887 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:39.887 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:39.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:39.982 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:39.983+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:40.043 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:40.043 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:40.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:40.092 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:40.094+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:40.184 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:40.185+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:40.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:40.284 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:40.286+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:40.313 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:40.313 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:40.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:40.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:17:40.385 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:40.386+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:40.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:40.453 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:40.453 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:40.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:40.507 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:40.509+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:40.586 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:40.588+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:40.586 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:40.604 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:40.621 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.046833055999997, 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.046813716000003, 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.046409316000002, 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.044714046000003, 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.044683513000003, 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.040282613999999, 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.040262132999999, 2024-09-16T22:17:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.040241557000002, 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.040217071999997, 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.040195500999999, 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034116902999997, 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034096570999999, 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034075352000002, 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034053980000003, 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034033141999998, 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.034012560999997, 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.033990994, 2024-09-16T22:17:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.029344119999998, 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9740387189999993, 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:17:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2968497360000004, 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:40.715 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:40.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:40.717 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:40.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:40.718 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:40.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:40.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:40.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:40.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:40.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:40.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:40.720 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:40.718+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:40.817 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:40.819+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:40.867 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:40.867 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:40.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:40.918 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:40.920+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:41.018 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:41.020+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:41.022 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:41.023 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:41.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:41.119 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:41.121+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:41.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.753248690000007, 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.918443637999999, 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.918422798999998, 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.918399559000001, 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917697674000003, 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.914565226999997, 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.914155616999999, 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.913869439999999, 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.913572215999999, 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.912783973000003, 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.912669203999997, 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.912417986999998, 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.912109629, 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.912070692, 2024-09-16T22:17:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.911785272000003, 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.905933769000001, 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.905909371, 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.905858721000001, 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.905825024000002, 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.905714054000001, 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:41.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:41.266 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:41.268+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:41.367 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:41.369+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:41.377 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:41.378 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:41.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:41.389 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:17:41.467 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:41.469+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:41.496 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:41.496 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:41.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:41.589 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:41.591+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:41.651 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.062005573, 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:41.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:41.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:41.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061966998999999, 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:41.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:41.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:41.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061947350000001, 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:41.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061905936999999, 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061870086999999, 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061852164000001, 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:41.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:41.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061832583000001, 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:41.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061807072000001, 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:41.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:41.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.06178869, 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061770150000001, 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.06175197, 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061734403999999, 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:41.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.699 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061712442000001, 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.700 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061694517999999, 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.704 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:41.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061677008, 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:41.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:41.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061659336999998, 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061217396, 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:41.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061102098999999, 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.0610751, 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:41.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:41.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.061021723, 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:41.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:41.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:41.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:41.731 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:41.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:41.733 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:41.734+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:41.833 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:41.835+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:41.876 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:41.876 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:41.877 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:41.877 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:41.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:41.969 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:41.971+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:42.027 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:42.027 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:42.028 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:42.028 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:42.028 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:42.035 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:42.036+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:42.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:42.136 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:42.137+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:42.141 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:42.141 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:42.141 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-16T22:17:42.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-16T22:17:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:42.181 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.676662954999998, 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675117178000001, 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:42.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675099097999997, 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:42.191 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:42.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675082535999998, 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:42.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:42.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:42.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675066047999998, 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:42.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675048975999999, 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:42.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.209 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675032469999998, 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.675002677999998, 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:42.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670670420999997, 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:42.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:42.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670648159999999, 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:42.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:42.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670627166000003, 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:42.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670606417999998, 2024-09-16T22:17:42.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:42.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670586116000003, 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:42.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670566113, 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670545988000001, 2024-09-16T22:17:42.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.248 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670526393000003, 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670505470999998, 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:42.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:42.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670484682999998, 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:42.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670463022, 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:42.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.670433932000002, 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:42.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.270 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:42.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:42.272 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:42.273+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:42.291 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:42.291 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:42.291 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:42.291 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:42.292 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:42.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:42.295 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:42.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-16T22:17:42.372 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:42.374+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:42.421 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:42.421 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:42.421 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:42.422 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:42.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:42.473 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:42.474+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:42.489 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:42.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:42.491 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:42.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-16T22:17:42.573 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:42.575+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:42.575 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:42.576 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:42.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:42.628 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:42.628 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:42.628 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:42.628 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:42.629 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:42.631 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:42.631 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:42.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-16T22:17:42.675 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:42.677+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:42.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:42.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.546471005000001, 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:42.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:17:42.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:17:42.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:17:42.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.495956316000004, 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:17:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:17:42.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:17:42.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.482630225999998, 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 57.427781701000001, 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:17:42.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:17:42.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:17:42.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:42.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4541716100000004, 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:17:42.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:17:42.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:17:42.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4541407629999998, 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:17:42.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:17:42.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4541043499999997, 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:17:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4540779370000001, 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:17:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:17:42.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.454052978, 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4540341229999996, 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4537189430000002, 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:17:42.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4537035669999998, 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:17:42.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4530884799999999, 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:17:42.789 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:17:42.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4530616390000004, 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:17:42.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:17:42.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4530375690000001, 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:17:42.798 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:17:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4528114270000003, 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:17:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4519900300000002, 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4519681699999998, 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:17:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4515814430000002, 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:17:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.4505885540000003, 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:17:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:42.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:42.827 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:42.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:42.828 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:42.828+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:42.829 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:42.829 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:42.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-16T22:17:42.847 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:42.848 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:42.865 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:42.865 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:42.881 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:42.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:42.915 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:42.927 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:42.929+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:42.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:42.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:42.951 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:42.966 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:42.966 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:42.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:42.969 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:42.970 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:42.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-16T22:17:42.974 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:42.974 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:42.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:43.066 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:43.068+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:43.125 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.126 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.129 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:43.130+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:43.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:43.158 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:43.158 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:43.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-16T22:17:43.229 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:43.231+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:43.279 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.279 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:43.330 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:43.332+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:43.335 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:43.335 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:43.335 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:43.336 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:43.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:43.339 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:43.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-16T22:17:43.400 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.400 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:17:43.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:43.431 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:43.433+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-16T22:17:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-16T22:17:43.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T22:17:43.510 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T22:17:43.532 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:43.534+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:43.580 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.580 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:43.634 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:43.636+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:43.697 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.697 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:43.746 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:43.748+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:43.835 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:43.837+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:43.852 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.853 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:43.935 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:43.937+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:43.967 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:43.968 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:43.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:44.037 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:44.038+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:44.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:44.128 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:17:44.137 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:44.139+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:44.210 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.210 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:44.238 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:44.240+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:44.327 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.327 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:44.338 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:44.340+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:44.440 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:44.441+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:44.444 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.444 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:44.541 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:44.543+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.601 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:44.642 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:44.644+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:44.716 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.716 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:44.742 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:44.744+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:44.832 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.833 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:44.843 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:44.844+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:44.944 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:44.945+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:44.948 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:44.948 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:44.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:44.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:45.045 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:45.047+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:45.101 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:45.101 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:45.104 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:45.104 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:45.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:45.121 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:45.121 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:45.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:45.142 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:45.151 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:45.153+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:45.159 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:45.178 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:45.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:45.197 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:45.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:45.246 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:45.248+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672694566000001, 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672655991999999, 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672636343000001, 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672594929999999, 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672559079999999, 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672541157000001, 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672521576000001, 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672496065000001, 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672477683, 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672459143000001, 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672440963, 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672423396999999, 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672401435000001, 2024-09-16T22:17:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672383511, 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672366001, 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.672348329999998, 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.671906389, 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.671791091999999, 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.671764093, 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.671710716, 2024-09-16T22:17:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.343 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:45.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:45.347 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:45.349+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:45.448 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:45.449+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:45.487 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:45.488 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:45.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:45.548 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:45.550+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:45.598 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:45.598 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:45.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:45.650 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:45.652+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:45.710 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.205766726, 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204220949000003, 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204202869, 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204186307000001, 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204169819000001, 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204152747000002, 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204136241, 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.204106449000001, 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199774192, 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199751931000002, 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199730936999998, 2024-09-16T22:17:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199710189000001, 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199689886999998, 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199669884000002, 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199649759000003, 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199630163999998, 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199609242000001, 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199588454000001, 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199566793000002, 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.199537702999997, 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:45.803 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:45.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:45.805 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:45.806+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:45.905 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:45.907+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:45.948 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:45.949 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:45.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:46.006 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:46.008+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:46.100 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:46.100 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:46.107 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:46.108+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:46.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:46.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:17:46.208 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:46.209+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:46.255 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.073067421000005, 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:17:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:17:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:17:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.022552731999994, 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:17:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:17:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.009226642000002, 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:17:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:17:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.954378116999997, 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:17:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:17:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:17:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:17:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9807680259999998, 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:17:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9807371790000001, 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:17:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.980700766, 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:17:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9806743529999995, 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:17:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9806493940000003, 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:17:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9806305389999999, 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9803153590000004, 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9802999830000001, 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:17:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:17:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9796848960000002, 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:17:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9796580549999998, 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:17:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9796339849999995, 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9794078430000006, 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:17:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9785864459999996, 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:17:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:17:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9785645859999992, 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:17:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:17:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:17:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9781778590000005, 2024-09-16T22:17:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:17:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:17:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9771849699999997, 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:17:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:17:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:17:46.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.354 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:46.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:46.356 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:46.357+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:46.457 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:46.459+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:46.500 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:46.500 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:46.500 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:46.500 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:46.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:46.558 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:46.560+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:46.613 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:46.613 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:46.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:46.663 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:46.665+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:46.761 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:46.762+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:46.766 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:46.766 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:46.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991479263999999, 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991459104999997, 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:46.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991440064000003, 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:17:46.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991404656999997, 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:17:46.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991385495000003, 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.789 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991366212999999, 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.793 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991347515000001, 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:17:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991329778999997, 2024-09-16T22:17:46.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:46.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991312149000002, 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:17:46.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:17:46.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.99129293, 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:17:46.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991275829999999, 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991257093999998, 2024-09-16T22:17:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:17:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991233246999997, 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:17:46.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991214268999997, 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991195153, 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:46.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:17:46.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:17:46.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991164900000001, 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:17:46.834 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:17:46.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:17:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.991144099000003, 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:17:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:17:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.965567346, 2024-09-16T22:17:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:17:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.965549768000002, 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:17:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.964532204000001, 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:46.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:46.861 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:46.863+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:46.963 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:46.965+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:46.969 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:46.969 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:46.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:47.064 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:47.066+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:47.084 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:47.084 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:47.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:47.133 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:17:47.165 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:47.166+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.155311699000002, 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.567087557000001, 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:17:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830871569999999, 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830851146000001, 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:17:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830831139000001, 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:17:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830810854999999, 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830640618, 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:17:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830610726, 2024-09-16T22:17:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:17:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:17:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830587089, 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830565024, 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:17:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830544094, 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:17:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.83051725, 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:17:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830495883999999, 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830475089, 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830448381, 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:17:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.83042219, 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:17:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830400458, 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:17:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830379527, 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830358991000001, 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:17:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.830326461, 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:17:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:17:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:17:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:47.309 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:47.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:47.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:47.311 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:47.313+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:47.412 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:47.414+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:47.460 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:47.460 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:47.460 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:47.460 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:47.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:47.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:47.476 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:47.494 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:47.511 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:47.513 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:47.515+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:47.527 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:47.527 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:47.544 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:47.544 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:47.560 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:47.577 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:47.577 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:47.578 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:47.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:47.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:47.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:47.614 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:47.615+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:47.701 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:47.701 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:47.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:47.755 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:47.757+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:47.815 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:47.818+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:47.818 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:47.818 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:47.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:47.917 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:47.918+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:47.974 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:47.974 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:47.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:48.017 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:48.019+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:48.119 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:48.120+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:48.131 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.131 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:48.219 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:48.221+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:48.264 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.264 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:48.321 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:48.322+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:48.381 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.381 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:48.422 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:48.424+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:48.510 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:48.510 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:48.510 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T22:17:48.511 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:48.511 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:48.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:48.556 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:48.557+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:48.618 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:48.618 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-16T22:17:48.618 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:48.623 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:48.625+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:48.661 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:48.724 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:48.725+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:48.812 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.812 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:48.864 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:48.866+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:48.923 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:48.924 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:48.926 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:48.927+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:48.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:49.027 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:49.029+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:49.076 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.076 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:49.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:17:49.168 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:49.170+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:49.228 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:49.230+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:49.240 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.240 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:49.328 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:49.330+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:49.392 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.392 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:49.430 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:49.431+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:49.504 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.504 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:49.530 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:49.533+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:49.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:49.615 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.615 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:49.667 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:49.669+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:49.728 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.729 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.732 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:49.733+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:49.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:49.833 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:49.835+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:49.879 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.879 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T22:17:49.911+0000 7f5db4d9f640 0 --2- 172.21.15.31:0/3801793288 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5db00a9700 0x7f5db013fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T22:17:49.933 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:49.934 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:49.935+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:49.950 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:49.950 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:49.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:49.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:49.982 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:49.983 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:49.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:49.999 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:50.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:50.015 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:50.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:50.034 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:50.036+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:50.049 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:50.049 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:50.135 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:50.137+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369920548000003, 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:17:50.145 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:50.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:17:50.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369900389000001, 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369881348, 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.154 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369845941000001, 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:17:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:17:50.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:17:50.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369826779, 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:17:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369807496999996, 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:17:50.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:17:50.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:17:50.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369788798999998, 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369771063000002, 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369753433, 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:50.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369734213999998, 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:50.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369717113999997, 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:17:50.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:17:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369698378000002, 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:17:50.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:17:50.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369674531000001, 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:17:50.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369655553000001, 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:17:50.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369636436999997, 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369606183999998, 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:17:50.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.369585383, 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.344008629999998, 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.343991052, 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.342973487999998, 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:50.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:50.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:17:50.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:17:50.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:50.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:50.228 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:50.229 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:17:50.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:50.273 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:50.275+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:50.336 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:50.338+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:50.341 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:50.342 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:50.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:50.437 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:50.440+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:50.498 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:50.498 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:50.499 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:50.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:50.551 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:50.553+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.554938790999998, 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:17:50.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:17:50.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:17:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.966714649, 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:17:50.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230498662, 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:17:50.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230478238, 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:50.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:17:50.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230458231, 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:17:50.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:50.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230437947, 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:50.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:17:50.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.23026771, 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:17:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:17:50.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230237818000001, 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230214180999999, 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230192116, 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:50.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230171186, 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:17:50.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:17:50.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230144341999999, 2024-09-16T22:17:50.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:17:50.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:17:50.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230122976000001, 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230102180999999, 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:17:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:17:50.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:17:50.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230075472999999, 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:17:50.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230049282, 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:17:50.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230027550000001, 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:17:50.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230006618999999, 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:17:50.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:17:50.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.229986083, 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.229953553, 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:50.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:17:50.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:17:50.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:50.707 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:50.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:50.709 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:50.711+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:50.809 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:50.811+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:50.857 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:50.857 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:50.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:50.909 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:50.911+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:50.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:50.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:50.973 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:50.973 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:50.973 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:50.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:51.025 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:51.027+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:51.090 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.090 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:51.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509420592, 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.091 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:51.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:17:51.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.50940085, 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:17:51.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509371399999999, 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:17:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:17:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509345773, 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:17:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:17:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:17:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509323119999999, 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:17:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:17:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:17:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509300142000001, 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:17:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:17:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:17:51.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509277088999999, 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509255369, 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:17:51.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:17:51.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509235165, 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509216574, 2024-09-16T22:17:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:17:51.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:17:51.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.509195044, 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.504075948000001, 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.504055679, 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.504033549000001, 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.504014068, 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.503989728000001, 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.503969762000001, 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.503948351, 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:17:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.503926590000001, 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:17:51.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:17:51.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.503906122, 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.175 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:51.176 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:51.178+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:51.277 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:51.278+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:51.334 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:51.334 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:51.335 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:51.385 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:51.386+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:51.478 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:51.480+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:51.490 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:51.490 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:51.579 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:51.581+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:51.647 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:51.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.071248044000001, 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.071228703999999, 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:51.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:51.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:51.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:51.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.070824303999999, 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:51.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:51.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.069129033999999, 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.069098500999999, 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:51.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:51.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.064697602000003, 2024-09-16T22:17:51.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:51.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:51.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:51.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.064677121000003, 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:51.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.064656544999998, 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:51.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.064632060000001, 2024-09-16T22:17:51.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.064610489000003, 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058531891000001, 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058511559000003, 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:51.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:51.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058490339999999, 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058468968, 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:51.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:51.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058448130000002, 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058427549000001, 2024-09-16T22:17:51.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:51.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.058405981999996, 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.053759108000001, 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:51.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:51.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.998453706999999, 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:17:51.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:17:51.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.321264724000001, 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:51.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:51.735 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:51.736+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:51.836 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:51.838+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:51.842 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:51.842 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:51.937 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:51.939+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:51.957 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:51.957 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:51.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:52.038 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:52.040+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:52.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.649838909, 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.815033857000003, 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.815013018000002, 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.814989777999998, 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:52.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.814287892999999, 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:52.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.811155446000001, 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.810745836000002, 2024-09-16T22:17:52.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:52.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.810459659000003, 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.810162435000002, 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.809374192, 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.809259423, 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.809008206000001, 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.808699848000003, 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.808660910999997, 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.808375491, 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.802523987999997, 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.802499589999996, 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.802448939999998, 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.802415242999999, 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.802304272999997, 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:52.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:52.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:52.161 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:52.162+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:52.198 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:52.198 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:52.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:52.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:17:52.240 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:52.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:52.261 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:52.263+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:52.262 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:52.262 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:52.278 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:52.278 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:52.281 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:52.281 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:52.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:52.298 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:52.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:52.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:52.334 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:52.362 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:52.364+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:52.391 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:52.391 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:52.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:52.484 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:52.486+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:52.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:52.593 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:52.595+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:52.664 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:52.667+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:52.696 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:52.696 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:52.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:52.765 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:52.767+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:52.807 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:52.807 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:52.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:52.866 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:52.868+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:52.918 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:52.919 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:52.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:52.967 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:52.968+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:53.067 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:53.069+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:53.070 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.070 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:53.167 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:53.169+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:53.172 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:17:53.184 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.184 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:53.269 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:53.270+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:53.340 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.340 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:53.370 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:53.371+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:53.453 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.453 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:53.470 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:53.472+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:53.565 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.565 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.571 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:53.573+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:53.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:53.628 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:53.628 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:53.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:17:53.672 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:53.673+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:53.718 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.718 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:53.773 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:53.774+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:53.836 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.836 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:53.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:53.874 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:53.876+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:17:53.936 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:17:53.948 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-16T22:17:53.179176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":707602049,"num_objects":2623,"num_object_clones":298,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":1893,"num_read":257346,"num_read_kb":906238,"num_write":20684,"num_write_kb":589668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19058,"ondisk_log_size":19058,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":595864,"kb_used_data":268516,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754378856,"statfs":{"total":773094113280,"available":772483948544,"internally_reserved":0,"allocated":274960384,"data_stored":365967385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":333499953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-79082712,"num_objects":-255,"num_object_clones":-34,"num_object_copies":-510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-255,"num_whiteouts":-24,"num_read":2258,"num_read_kb":1918,"num_write":374,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.050253"},"pg_stats":[{"pgid":"3.1f","version":"103'717","reported_seq":5365,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.238372+0000","last_change":"2024-09-16T22:17:51.236608+0000","last_active":"2024-09-16T22:17:51.238372+0000","last_peered":"2024-09-16T22:17:51.238372+0000","last_clean":"2024-09-16T22:17:51.238372+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:17:51.238372+0000","last_undegraded":"2024-09-16T22:17:51.238372+0000","last_fullsized":"2024-09-16T22:17:51.238372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:29.069777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929411,"stat_sum":{"num_bytes":20943678,"num_objects":75,"num_object_clones":6,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":58,"num_read":7291,"num_read_kb":45433,"num_write":852,"num_write_kb":6331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.1e","version":"103'640","reported_seq":5198,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.283538+0000","last_change":"2024-09-16T22:17:51.236940+0000","last_active":"2024-09-16T22:17:51.283538+0000","last_peered":"2024-09-16T22:17:51.283538+0000","last_clean":"2024-09-16T22:17:51.283538+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:17:51.283538+0000","last_undegraded":"2024-09-16T22:17:51.283538+0000","last_fullsized":"2024-09-16T22:17:51.283538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:51.782707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021829890000000002,"stat_sum":{"num_bytes":20532938,"num_objects":73,"num_object_clones":5,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":54,"num_read":7239,"num_read_kb":27094,"num_write":856,"num_write_kb":35886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.1d","version":"103'668","reported_seq":2669,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.248895+0000","last_change":"2024-09-16T22:17:51.166107+0000","last_active":"2024-09-16T22:17:51.248895+0000","last_peered":"2024-09-16T22:17:51.248895+0000","last_clean":"2024-09-16T22:17:51.248895+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:17:51.248895+0000","last_undegraded":"2024-09-16T22:17:51.248895+0000","last_fullsized":"2024-09-16T22:17:51.248895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:45.951664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069119999999999,"stat_sum":{"num_bytes":15885769,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":44,"num_read":3877,"num_read_kb":26708,"num_write":890,"num_write_kb":11238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.1c","version":"103'469","reported_seq":5767,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.280016+0000","last_change":"2024-09-16T22:17:51.170347+0000","last_active":"2024-09-16T22:17:51.280016+0000","last_peered":"2024-09-16T22:17:51.280016+0000","last_clean":"2024-09-16T22:17:51.280016+0000","last_became_active":"2024-09-16T22:16:15.135556+0000","last_became_peered":"2024-09-16T22:16:15.135556+0000","last_unstale":"2024-09-16T22:17:51.280016+0000","last_undegraded":"2024-09-16T22:17:51.280016+0000","last_fullsized":"2024-09-16T22:17:51.280016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:09.896742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061639900000000001,"stat_sum":{"num_bytes":12085607,"num_objects":77,"num_object_clones":8,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":58,"num_read":6101,"num_read_kb":14834,"num_write":409,"num_write_kb":37524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.1b","version":"103'535","reported_seq":6750,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.241011+0000","last_change":"2024-09-16T22:17:51.167912+0000","last_active":"2024-09-16T22:17:51.241011+0000","last_peered":"2024-09-16T22:17:51.241011+0000","last_clean":"2024-09-16T22:17:51.241011+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:17:51.241011+0000","last_undegraded":"2024-09-16T22:17:51.241011+0000","last_fullsized":"2024-09-16T22:17:51.241011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:36.749419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057646499999999996,"stat_sum":{"num_bytes":26611988,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":46,"num_read":7435,"num_read_kb":20983,"num_write":513,"num_write_kb":20827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.1a","version":"103'555","reported_seq":21631,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.291015+0000","last_change":"2024-09-16T22:17:51.170094+0000","last_active":"2024-09-16T22:17:51.291015+0000","last_peered":"2024-09-16T22:17:51.291015+0000","last_clean":"2024-09-16T22:17:51.291015+0000","last_became_active":"2024-09-16T22:16:15.126160+0000","last_became_peered":"2024-09-16T22:16:15.126160+0000","last_unstale":"2024-09-16T22:17:51.291015+0000","last_undegraded":"2024-09-16T22:17:51.291015+0000","last_fullsized":"2024-09-16T22:17:51.291015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:30.895616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038293300000000001,"stat_sum":{"num_bytes":42056256,"num_objects":91,"num_object_clones":12,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":62,"num_read":22935,"num_read_kb":52351,"num_write":490,"num_write_kb":17309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.19","version":"103'606","reported_seq":11386,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.253865+0000","last_change":"2024-09-16T22:17:51.168895+0000","last_active":"2024-09-16T22:17:51.253865+0000","last_peered":"2024-09-16T22:17:51.253865+0000","last_clean":"2024-09-16T22:17:51.253865+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:17:51.253865+0000","last_undegraded":"2024-09-16T22:17:51.253865+0000","last_fullsized":"2024-09-16T22:17:51.253865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:31:59.655594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013227239999999999,"stat_sum":{"num_bytes":29032326,"num_objects":81,"num_object_clones":8,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":57,"num_read":13835,"num_read_kb":34339,"num_write":529,"num_write_kb":15351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.18","version":"103'461","reported_seq":5540,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.237188+0000","last_change":"2024-09-16T22:17:51.237188+0000","last_active":"2024-09-16T22:17:51.237188+0000","last_peered":"2024-09-16T22:17:51.237188+0000","last_clean":"2024-09-16T22:17:51.237188+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:17:51.237188+0000","last_undegraded":"2024-09-16T22:17:51.237188+0000","last_fullsized":"2024-09-16T22:17:51.237188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:44.744703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021134819999999999,"stat_sum":{"num_bytes":13394506,"num_objects":76,"num_object_clones":5,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":61,"num_read":5464,"num_read_kb":14995,"num_write":456,"num_write_kb":8922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.17","version":"102'913","reported_seq":7992,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.235946+0000","last_change":"2024-09-16T22:17:51.236611+0000","last_active":"2024-09-16T22:17:51.235946+0000","last_peered":"2024-09-16T22:17:51.235946+0000","last_clean":"2024-09-16T22:17:51.235946+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:17:51.235946+0000","last_undegraded":"2024-09-16T22:17:51.235946+0000","last_fullsized":"2024-09-16T22:17:51.235946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:54.066122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001923077,"stat_sum":{"num_bytes":26327474,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":59,"num_read":9777,"num_read_kb":17495,"num_write":1256,"num_write_kb":30818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.16","version":"103'480","reported_seq":4590,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.305493+0000","last_change":"2024-09-16T22:17:51.165423+0000","last_active":"2024-09-16T22:17:51.305493+0000","last_peered":"2024-09-16T22:17:51.305493+0000","last_clean":"2024-09-16T22:17:51.305493+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:17:51.305493+0000","last_undegraded":"2024-09-16T22:17:51.305493+0000","last_fullsized":"2024-09-16T22:17:51.305493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:28:09.750573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079734900000000004,"stat_sum":{"num_bytes":31965024,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":55,"num_read":4753,"num_read_kb":21557,"num_write":533,"num_write_kb":35347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.15","version":"103'718","reported_seq":2451,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.234855+0000","last_change":"2024-09-16T22:17:51.174129+0000","last_active":"2024-09-16T22:17:51.234855+0000","last_peered":"2024-09-16T22:17:51.234855+0000","last_clean":"2024-09-16T22:17:51.234855+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:17:51.234855+0000","last_undegraded":"2024-09-16T22:17:51.234855+0000","last_fullsized":"2024-09-16T22:17:51.234855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:49.088375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009922939999999999,"stat_sum":{"num_bytes":21825834,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":61,"num_read":3550,"num_read_kb":14283,"num_write":841,"num_write_kb":23246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.14","version":"103'357","reported_seq":10030,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.165665+0000","last_change":"2024-09-16T22:17:51.165665+0000","last_active":"2024-09-16T22:17:51.165665+0000","last_peered":"2024-09-16T22:17:51.165665+0000","last_clean":"2024-09-16T22:17:51.165665+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:17:51.165665+0000","last_undegraded":"2024-09-16T22:17:51.165665+0000","last_fullsized":"2024-09-16T22:17:51.165665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:20.137460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014607999999999999,"stat_sum":{"num_bytes":35220268,"num_objects":88,"num_object_clones":13,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":62,"num_read":9648,"num_read_kb":39690,"num_write":305,"num_write_kb":14214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.163806+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:17:51.163806+0000","last_peered":"2024-09-16T22:17:51.163806+0000","last_clean":"2024-09-16T22:17:51.163806+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:17:51.163806+0000","last_undegraded":"2024-09-16T22:17:51.163806+0000","last_fullsized":"2024-09-16T22:17:51.163806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"103'808","reported_seq":12970,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.262586+0000","last_change":"2024-09-16T22:17:51.167808+0000","last_active":"2024-09-16T22:17:51.262586+0000","last_peered":"2024-09-16T22:17:51.262586+0000","last_clean":"2024-09-16T22:17:51.262586+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:17:51.262586+0000","last_undegraded":"2024-09-16T22:17:51.262586+0000","last_fullsized":"2024-09-16T22:17:51.262586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:46.043604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047211699999999998,"stat_sum":{"num_bytes":14713706,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":51,"num_read":15848,"num_read_kb":33561,"num_write":856,"num_write_kb":29256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":179,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.167310+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:17:51.167310+0000","last_peered":"2024-09-16T22:17:51.167310+0000","last_clean":"2024-09-16T22:17:51.167310+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:17:51.167310+0000","last_undegraded":"2024-09-16T22:17:51.167310+0000","last_fullsized":"2024-09-16T22:17:51.167310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"100'730","reported_seq":4694,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.168323+0000","last_change":"2024-09-16T22:17:51.168323+0000","last_active":"2024-09-16T22:17:51.168323+0000","last_peered":"2024-09-16T22:17:51.168323+0000","last_clean":"2024-09-16T22:17:51.168323+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:17:51.168323+0000","last_undegraded":"2024-09-16T22:17:51.168323+0000","last_fullsized":"2024-09-16T22:17:51.168323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.985176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092312600000000002,"stat_sum":{"num_bytes":27069355,"num_objects":93,"num_object_clones":8,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":65,"num_read":6147,"num_read_kb":17750,"num_write":905,"num_write_kb":25285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":182,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.164522+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:17:51.164522+0000","last_peered":"2024-09-16T22:17:51.164522+0000","last_clean":"2024-09-16T22:17:51.164522+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:17:51.164522+0000","last_undegraded":"2024-09-16T22:17:51.164522+0000","last_fullsized":"2024-09-16T22:17:51.164522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"103'442","reported_seq":13488,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.239307+0000","last_change":"2024-09-16T22:17:51.235377+0000","last_active":"2024-09-16T22:17:51.239307+0000","last_peered":"2024-09-16T22:17:51.239307+0000","last_clean":"2024-09-16T22:17:51.239307+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:17:51.239307+0000","last_undegraded":"2024-09-16T22:17:51.239307+0000","last_fullsized":"2024-09-16T22:17:51.239307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:36.203408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068984799999999996,"stat_sum":{"num_bytes":19267148,"num_objects":82,"num_object_clones":9,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":59,"num_read":13186,"num_read_kb":25115,"num_write":437,"num_write_kb":19448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":181,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.234432+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:17:51.234432+0000","last_peered":"2024-09-16T22:17:51.234432+0000","last_clean":"2024-09-16T22:17:51.234432+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:17:51.234432+0000","last_undegraded":"2024-09-16T22:17:51.234432+0000","last_fullsized":"2024-09-16T22:17:51.234432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"101'419","reported_seq":2260,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.164546+0000","last_change":"2024-09-16T22:17:51.164812+0000","last_active":"2024-09-16T22:17:51.164546+0000","last_peered":"2024-09-16T22:17:51.164546+0000","last_clean":"2024-09-16T22:17:51.164546+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:17:51.164546+0000","last_undegraded":"2024-09-16T22:17:51.164546+0000","last_fullsized":"2024-09-16T22:17:51.164546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:07.976995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047941299999999999,"stat_sum":{"num_bytes":43310036,"num_objects":85,"num_object_clones":13,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":58,"num_read":2401,"num_read_kb":23982,"num_write":422,"num_write_kb":9709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.3","version":"103'813","reported_seq":8151,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.305045+0000","last_change":"2024-09-16T22:17:51.173550+0000","last_active":"2024-09-16T22:17:51.305045+0000","last_peered":"2024-09-16T22:17:51.305045+0000","last_clean":"2024-09-16T22:17:51.305045+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:17:51.305045+0000","last_undegraded":"2024-09-16T22:17:51.305045+0000","last_fullsized":"2024-09-16T22:17:51.305045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:06.628179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000410128,"stat_sum":{"num_bytes":22643519,"num_objects":109,"num_object_clones":14,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":78,"num_read":9626,"num_read_kb":22932,"num_write":841,"num_write_kb":8389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":184,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.163982+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:17:51.163982+0000","last_peered":"2024-09-16T22:17:51.163982+0000","last_clean":"2024-09-16T22:17:51.163982+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:17:51.163982+0000","last_undegraded":"2024-09-16T22:17:51.163982+0000","last_fullsized":"2024-09-16T22:17:51.163982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"103'341","reported_seq":10853,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.236904+0000","last_change":"2024-09-16T22:17:51.236904+0000","last_active":"2024-09-16T22:17:51.236904+0000","last_peered":"2024-09-16T22:17:51.236904+0000","last_clean":"2024-09-16T22:17:51.236904+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:17:51.236904+0000","last_undegraded":"2024-09-16T22:17:51.236904+0000","last_fullsized":"2024-09-16T22:17:51.236904+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:23.013755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021880559999999999,"stat_sum":{"num_bytes":16647264,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":49,"num_read":10739,"num_read_kb":63502,"num_write":279,"num_write_kb":14390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T22:17:49.134908+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:17:49.134908+0000","last_peered":"2024-09-16T22:17:49.134908+0000","last_clean":"2024-09-16T22:17:49.134908+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:17:49.134908+0000","last_undegraded":"2024-09-16T22:17:49.134908+0000","last_fullsized":"2024-09-16T22:17:49.134908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"103'561","reported_seq":15951,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.235589+0000","last_change":"2024-09-16T22:17:51.235589+0000","last_active":"2024-09-16T22:17:51.235589+0000","last_peered":"2024-09-16T22:17:51.235589+0000","last_clean":"2024-09-16T22:17:51.235589+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:17:51.235589+0000","last_undegraded":"2024-09-16T22:17:51.235589+0000","last_fullsized":"2024-09-16T22:17:51.235589+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:02.478308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073965599999999999,"stat_sum":{"num_bytes":16454664,"num_objects":74,"num_object_clones":9,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":52,"num_read":16079,"num_read_kb":102607,"num_write":663,"num_write_kb":44003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":182,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.163499+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:17:51.163499+0000","last_peered":"2024-09-16T22:17:51.163499+0000","last_clean":"2024-09-16T22:17:51.163499+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:17:51.163499+0000","last_undegraded":"2024-09-16T22:17:51.163499+0000","last_fullsized":"2024-09-16T22:17:51.163499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"103'328","reported_seq":2527,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.168073+0000","last_change":"2024-09-16T22:17:51.168073+0000","last_active":"2024-09-16T22:17:51.168073+0000","last_peered":"2024-09-16T22:17:51.168073+0000","last_clean":"2024-09-16T22:17:51.168073+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:17:51.168073+0000","last_undegraded":"2024-09-16T22:17:51.168073+0000","last_fullsized":"2024-09-16T22:17:51.168073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:10.580397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073524199999999999,"stat_sum":{"num_bytes":25370039,"num_objects":81,"num_object_clones":8,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":59,"num_read":2105,"num_read_kb":24528,"num_write":328,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"2.3","version":"20'1","reported_seq":183,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.164001+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:17:51.164001+0000","last_peered":"2024-09-16T22:17:51.164001+0000","last_clean":"2024-09-16T22:17:51.164001+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:17:51.164001+0000","last_undegraded":"2024-09-16T22:17:51.164001+0000","last_fullsized":"2024-09-16T22:17:51.164001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":272,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.169540+0000","last_change":"2024-09-16T22:15:58.402712+0000","last_active":"2024-09-16T22:17:51.169540+0000","last_peered":"2024-09-16T22:17:51.169540+0000","last_clean":"2024-09-16T22:17:51.169540+0000","last_became_active":"2024-09-16T22:15:58.401744+0000","last_became_peered":"2024-09-16T22:15:58.401744+0000","last_unstale":"2024-09-16T22:17:51.169540+0000","last_undegraded":"2024-09-16T22:17:51.169540+0000","last_fullsized":"2024-09-16T22:17:51.169540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:15.946469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"100'1046","reported_seq":3972,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.167969+0000","last_change":"2024-09-16T22:17:51.168157+0000","last_active":"2024-09-16T22:17:51.167969+0000","last_peered":"2024-09-16T22:17:51.167969+0000","last_clean":"2024-09-16T22:17:51.167969+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:17:51.167969+0000","last_undegraded":"2024-09-16T22:17:51.167969+0000","last_fullsized":"2024-09-16T22:17:51.167969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:45.697562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081079500000000001,"stat_sum":{"num_bytes":34699590,"num_objects":94,"num_object_clones":12,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":65,"num_read":7535,"num_read_kb":19577,"num_write":1310,"num_write_kb":26555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.9","version":"103'479","reported_seq":4809,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.164767+0000","last_change":"2024-09-16T22:17:51.164767+0000","last_active":"2024-09-16T22:17:51.164767+0000","last_peered":"2024-09-16T22:17:51.164767+0000","last_clean":"2024-09-16T22:17:51.164767+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:17:51.164767+0000","last_undegraded":"2024-09-16T22:17:51.164767+0000","last_fullsized":"2024-09-16T22:17:51.164767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:01.711648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084776000000000003,"stat_sum":{"num_bytes":11562241,"num_objects":80,"num_object_clones":5,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":57,"num_read":5339,"num_read_kb":19029,"num_write":495,"num_write_kb":6611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.a","version":"103'627","reported_seq":2341,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.289797+0000","last_change":"2024-09-16T22:17:51.165163+0000","last_active":"2024-09-16T22:17:51.289797+0000","last_peered":"2024-09-16T22:17:51.289797+0000","last_clean":"2024-09-16T22:17:51.289797+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:17:51.289797+0000","last_undegraded":"2024-09-16T22:17:51.289797+0000","last_fullsized":"2024-09-16T22:17:51.289797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:34.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099207200000000009,"stat_sum":{"num_bytes":16797990,"num_objects":90,"num_object_clones":4,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":69,"num_read":3615,"num_read_kb":7458,"num_write":610,"num_write_kb":3300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.b","version":"100'411","reported_seq":14240,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T22:17:49.135329+0000","last_change":"2024-09-16T22:17:49.135812+0000","last_active":"2024-09-16T22:17:49.135329+0000","last_peered":"2024-09-16T22:17:49.135329+0000","last_clean":"2024-09-16T22:17:49.135329+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:17:49.135329+0000","last_undegraded":"2024-09-16T22:17:49.135329+0000","last_fullsized":"2024-09-16T22:17:49.135329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:11.995960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077852499999999996,"stat_sum":{"num_bytes":33929740,"num_objects":93,"num_object_clones":10,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":62,"num_read":14320,"num_read_kb":47243,"num_write":345,"num_write_kb":12203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.c","version":"103'406","reported_seq":3073,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.173909+0000","last_change":"2024-09-16T22:17:51.173909+0000","last_active":"2024-09-16T22:17:51.173909+0000","last_peered":"2024-09-16T22:17:51.173909+0000","last_clean":"2024-09-16T22:17:51.173909+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:17:51.173909+0000","last_undegraded":"2024-09-16T22:17:51.173909+0000","last_fullsized":"2024-09-16T22:17:51.173909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:02.624504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077581899999999997,"stat_sum":{"num_bytes":31433469,"num_objects":89,"num_object_clones":15,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":65,"num_read":2728,"num_read_kb":16306,"num_write":345,"num_write_kb":8501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.d","version":"103'726","reported_seq":2960,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.237784+0000","last_change":"2024-09-16T22:17:51.166053+0000","last_active":"2024-09-16T22:17:51.237784+0000","last_peered":"2024-09-16T22:17:51.237784+0000","last_clean":"2024-09-16T22:17:51.237784+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:17:51.237784+0000","last_undegraded":"2024-09-16T22:17:51.237784+0000","last_fullsized":"2024-09-16T22:17:51.237784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:54:44.722185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043096999999999998,"stat_sum":{"num_bytes":4849113,"num_objects":72,"num_object_clones":6,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":58,"num_read":5601,"num_read_kb":8534,"num_write":758,"num_write_kb":7999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.e","version":"103'577","reported_seq":6363,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.241097+0000","last_change":"2024-09-16T22:17:51.165327+0000","last_active":"2024-09-16T22:17:51.241097+0000","last_peered":"2024-09-16T22:17:51.241097+0000","last_clean":"2024-09-16T22:17:51.241097+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:17:51.241097+0000","last_undegraded":"2024-09-16T22:17:51.241097+0000","last_fullsized":"2024-09-16T22:17:51.241097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:18.911270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000429157,"stat_sum":{"num_bytes":17939090,"num_objects":73,"num_object_clones":6,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":56,"num_read":6774,"num_read_kb":60009,"num_write":586,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.f","version":"102'736","reported_seq":5287,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.271859+0000","last_change":"2024-09-16T22:17:51.173546+0000","last_active":"2024-09-16T22:17:51.271859+0000","last_peered":"2024-09-16T22:17:51.271859+0000","last_clean":"2024-09-16T22:17:51.271859+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:17:51.271859+0000","last_undegraded":"2024-09-16T22:17:51.271859+0000","last_fullsized":"2024-09-16T22:17:51.271859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:00.655927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044158700000000001,"stat_sum":{"num_bytes":19830776,"num_objects":91,"num_object_clones":12,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":65,"num_read":7280,"num_read_kb":9251,"num_write":840,"num_write_kb":27000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.10","version":"102'409","reported_seq":12077,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.169864+0000","last_change":"2024-09-16T22:17:51.170041+0000","last_active":"2024-09-16T22:17:51.169864+0000","last_peered":"2024-09-16T22:17:51.169864+0000","last_clean":"2024-09-16T22:17:51.169864+0000","last_became_active":"2024-09-16T22:16:15.127331+0000","last_became_peered":"2024-09-16T22:16:15.127331+0000","last_unstale":"2024-09-16T22:17:51.169864+0000","last_undegraded":"2024-09-16T22:17:51.169864+0000","last_fullsized":"2024-09-16T22:17:51.169864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:25.289037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040261300000000002,"stat_sum":{"num_bytes":11570239,"num_objects":72,"num_object_clones":7,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":56,"num_read":11847,"num_read_kb":29918,"num_write":377,"num_write_kb":8003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.11","version":"103'1149","reported_seq":4540,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.302195+0000","last_change":"2024-09-16T22:17:51.165353+0000","last_active":"2024-09-16T22:17:51.302195+0000","last_peered":"2024-09-16T22:17:51.302195+0000","last_clean":"2024-09-16T22:17:51.302195+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:17:51.302195+0000","last_undegraded":"2024-09-16T22:17:51.302195+0000","last_fullsized":"2024-09-16T22:17:51.302195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1149,"log_dups_size":0,"ondisk_log_size":1149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:14.946180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001736196,"stat_sum":{"num_bytes":20596292,"num_objects":91,"num_object_clones":12,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":61,"num_read":8271,"num_read_kb":21636,"num_write":1498,"num_write_kb":8872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.12","version":"103'394","reported_seq":2330,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.235373+0000","last_change":"2024-09-16T22:17:51.235373+0000","last_active":"2024-09-16T22:17:51.235373+0000","last_peered":"2024-09-16T22:17:51.235373+0000","last_clean":"2024-09-16T22:17:51.235373+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:17:51.235373+0000","last_undegraded":"2024-09-16T22:17:51.235373+0000","last_fullsized":"2024-09-16T22:17:51.235373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:23.966868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073826999999999996,"stat_sum":{"num_bytes":15340915,"num_objects":90,"num_object_clones":8,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":74,"num_read":2181,"num_read_kb":17800,"num_write":313,"num_write_kb":6861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"a"}]},{"pgid":"3.13","version":"103'462","reported_seq":3117,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T22:17:51.165172+0000","last_change":"2024-09-16T22:17:51.165172+0000","last_active":"2024-09-16T22:17:51.165172+0000","last_peered":"2024-09-16T22:17:51.165172+0000","last_clean":"2024-09-16T22:17:51.165172+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:17:51.165172+0000","last_undegraded":"2024-09-16T22:17:51.165172+0000","last_fullsized":"2024-09-16T22:17:51.165172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:44.045327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202219999999999,"stat_sum":{"num_bytes":7104808,"num_objects":73,"num_object_clones":6,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":57,"num_read":3753,"num_read_kb":5683,"num_write":443,"num_write_kb":6334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"24","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":707011662,"num_objects":2619,"num_object_clones":298,"num_object_copies":5238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":1893,"num_read":257280,"num_read_kb":906183,"num_write":20581,"num_write_kb":588038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262774784,"data_stored":359005048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18983,"ondisk_log_size":18983,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60088,"kb_used_data":33060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94311752,"statfs":{"total":96636764160,"available":96575234048,"internally_reserved":0,"allocated":33853440,"data_stored":45766077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71388,"kb_used_data":26312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94300452,"statfs":{"total":96636764160,"available":96563662848,"internally_reserved":0,"allocated":26943488,"data_stored":39768229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72944,"kb_used_data":41756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298896,"statfs":{"total":96636764160,"available":96562069504,"internally_reserved":0,"allocated":42758144,"data_stored":54648291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84164,"kb_used_data":30772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94287676,"statfs":{"total":96636764160,"available":96550580224,"internally_reserved":0,"allocated":31510528,"data_stored":40281705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106548,"kb_used_data":55824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94265292,"statfs":{"total":96636764160,"available":96527659008,"internally_reserved":0,"allocated":57163776,"data_stored":70677252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45592,"kb_used_data":14396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326248,"statfs":{"total":96636764160,"available":96590077952,"internally_reserved":0,"allocated":14741504,"data_stored":18281802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":28580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":29265920,"data_stored":38491492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":38723584,"data_stored":58052537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27967488,"data_stored":37853502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12886016,"data_stored":17077925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55885824,"data_stored":70057963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":30240768,"data_stored":39668567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41541632,"data_stored":54082498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25133056,"data_stored":38612068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32632832,"data_stored":45200265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36487168,"data_stored":56452260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:17:53.975 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:53.976+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:53.992 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:53.992 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:54.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:54.085 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:54.087+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:54.164 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:54.165 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:54.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:54.176 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:54.177+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:54.276 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:54.278+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:54.324 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:54.325 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:54.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:54.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:54.377 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:54.379+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:54.441 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:54.441 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:54.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:54.478 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:54.480+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:54.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:54.520 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:54.537 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980688769, 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:17:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:17:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:17:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:17:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980669027000001, 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:17:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980639577000002, 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980613949999999, 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:17:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980591297, 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980568319, 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:17:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:17:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980545266, 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:17:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980523546000001, 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980503341999999, 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:17:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:17:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980484750999999, 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:17:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980463221000001, 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975344124999999, 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975323855999999, 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:17:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975301726000001, 2024-09-16T22:17:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975282244999999, 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:17:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:17:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975257904999999, 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:17:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975237938999999, 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975216528000001, 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:17:54.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975194767000001, 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:17:54.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.975174298999999, 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:54.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:17:54.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:17:54.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:54.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:54.651 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:54.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:54.653 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:54.654 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:54.654 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:54.654 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:54.654 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:54.655 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:54.656 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:54.654+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:54.753 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:54.755+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:54.801 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:54.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:54.801 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:54.801 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:54.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:54.854 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:54.856+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:54.916 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:54.916 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:54.916 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:54.916 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:54.917 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:54.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:54.966 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:54.968+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:55.055 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:55.057+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.498394380999997, 2024-09-16T22:17:55.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:17:55.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:17:55.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.498375041000003, 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.497970640999995, 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:55.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.496275370999996, 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.496244837999996, 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:17:55.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.491843939000006, 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:17:55.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.491823457999999, 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:55.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:17:55.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.491802882000002, 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:17:55.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.491778397000004, 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:17:55.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:17:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.491756826, 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485678227999998, 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485657896000006, 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:55.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485636677000002, 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:17:55.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:17:55.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:17:55.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485615304999996, 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:17:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:17:55.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485594466999999, 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:17:55.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485573885999997, 2024-09-16T22:17:55.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:17:55.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.485552319000007, 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:17:55.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.480905445000005, 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.425600043999999, 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:17:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:17:55.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.748411060999999, 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:55.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:17:55.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:55.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:55.162 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:55.164+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:55.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:17:55.262 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:55.263+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:55.322 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:55.322 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:55.322 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:17:55.322 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:55.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:55.364 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:55.365+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:55.439 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:55.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:17:55.440 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:17:55.440 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:17:55.440 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:55.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:55.465 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:55.466+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 112.130547265, 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:17:55.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:17:55.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:17:55.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.295742212999997, 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.295721374000003, 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.295698133999998, 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.294996249, 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:17:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:17:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.291863802000002, 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:17:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.291454192000003, 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:17:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:17:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:17:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.291168014999997, 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:17:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:17:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.290870791000003, 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.290082548000001, 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:17:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.289967779000001, 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:17:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.289716562000002, 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:17:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:17:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.289408203999997, 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.289369266999998, 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:17:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.289083847000001, 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:17:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.283232343999998, 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:17:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.283207945999997, 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:17:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.283157295999999, 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:17:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.283123599, 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:17:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.283012628999998, 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:17:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:17:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:17:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:17:55.649 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:17:55.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:55.651 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:55.653+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:55.752 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:55.753+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:55.771 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:55.771 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:55.771 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:55.771 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:55.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:55.853 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:55.855+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:55.930 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:55.931 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:55.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:55.983 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:55.984+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:56.043 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.454038509, 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:56.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453999934999999, 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:56.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453980285999997, 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:56.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:56.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453938872999998, 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453903023000002, 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:56.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:56.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:56.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453885100000001, 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:56.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:56.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453865518999997, 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453840008, 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453821626, 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:56.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:56.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453803086000001, 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:56.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453784906000003, 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.089 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453767339999999, 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.090 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:56.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453745378000001, 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:56.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453727454000003, 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453709944000003, 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:56.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453692273000001, 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:56.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:56.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453250332000003, 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:56.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453135035000003, 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453108036000003, 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:56.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.453054659000003, 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:56.122 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:56.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.126 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:56.128 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:56.128+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:17:56.227 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:56.228+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:56.277 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.278 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:56.278 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:56.278 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:56.328 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:56.329+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:56.429 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:56.430+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:56.429 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:56.429 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:56.430 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:56.530 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:56.531+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.077533191999997, 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075987415, 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:17:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075969335000003, 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075952772999997, 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:17:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:17:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075936284999997, 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075919212999999, 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:17:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:17:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075902706999997, 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:17:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.075872914999998, 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:17:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071540658000004, 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:17:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:17:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071518396999998, 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071497403000002, 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:17:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071476654999998, 2024-09-16T22:17:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071456353000002, 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:17:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:17:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071436349999999, 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:17:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071416225, 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:17:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071396630000002, 2024-09-16T22:17:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:17:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:17:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071375707999998, 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:17:56.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:17:56.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071354919999997, 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:17:56.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:17:56.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:56.665 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071333258999999, 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:17:56.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.071304169000001, 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:56.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:56.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:56.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:56.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:56.674 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:17:56.676 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:56.676+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:56.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:17:56.776 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:56.777+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:56.780 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.780 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:56.781 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:17:56.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:56.806 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:56.823 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:56.823 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:56.855 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:56.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:56.872 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:56.877 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:56.878+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:56.888 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:56.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:17:56.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:56.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:56.922 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:56.985 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:56.986+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:57.044 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.044 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:17:57.093 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:57.094+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:57.180 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:57.181+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:57.201 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.202 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:17:57.281 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:57.282+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:57.319 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.319 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:17:57.383 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:57.383+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:57.431 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.431 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:17:57.484 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:57.485+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:57.547 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.547 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:17:57.597 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:57.598+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:57.686 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:57.686+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:17:57.786 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:57.788+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:57.861 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.861 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:17:57.887 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:57.888+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:57.977 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:57.978 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:57.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:17:57.988 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:57.988+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:58.089 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:58.091+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:58.093 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.093 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:17:58.190 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:58.191+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:58.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:17:58.219 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.219 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:17:58.291 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:58.291+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:58.388 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.388 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.391 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:58.392+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:58.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:17:58.493 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:58.493+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:58.504 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.504 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:17:58.594 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:58.595+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:58.665 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.665 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:17:58.713 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:17:58.714+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:17:58.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:17:58.827 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:58.829+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:58.897 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:17:58.898+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:17:58.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:17:58.934 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:58.934 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:58.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:17:58.951 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:17:58.952 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:17:58.952 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T22:17:58.952 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:17:58.952 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:17:58.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T22:17:58.997 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:58.998+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:59.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:17:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:17:59.066 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.066 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-16T22:17:59.066 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.076 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:17:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:17:59.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:17:59.092 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:17:59.092 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:17:59.092 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:17:59.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:17:59.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:17:59.108 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:17:59.116 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:17:59.118+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:17:59.125 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:17:59.126 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:17:59.133 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:17:59.151 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:17:59.152 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:17:59.173 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:17:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:17:59.194 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:17:59.194 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:17:59.199 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:17:59.200+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:17:59.229 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:215632 3.6 3:62565073:::rbd_header.11cd13540b0c:head [call rbd.get_create_timestamp in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:59.232099+0000", 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.9163000000000001e-05, 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665799999999999, 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 215632 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232099+0000", 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232099+0000", 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232100+0000", 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8500000000000001e-07 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232103+0000", 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2540000000000001e-06 2024-09-16T22:17:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232111+0000", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3189999999999997e-06 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232114+0000", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8310000000000003e-06 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232143+0000", 2024-09-16T22:17:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8583999999999999e-05 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:59.232183+0000", 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0228999999999997e-05 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-16T22:17:59.234 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:17:59.300 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:59.302+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:59.383 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:59.383 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:59.384 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:59.384 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:17:59.435 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:17:59.436+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:17:59.496 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906856427000001, 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:17:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906817853, 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:17:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906798203999998, 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:17:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:17:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906756790999999, 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906720941000003, 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:17:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906703018000002, 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906683436999998, 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:17:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906657926000001, 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906639544000001, 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:17:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:17:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:17:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:17:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906621004000002, 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:17:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:17:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906602823999997, 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:17:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906585258, 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:17:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:17:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:17:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906563296000002, 2024-09-16T22:17:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:17:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906545371999997, 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906527861999997, 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:17:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:17:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906510191000002, 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:17:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:17:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:17:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.906068249999997, 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:17:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:17:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.905952953000003, 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:59.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:17:59.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.574 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.905925953999997, 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:59.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.905872576999997, 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:17:59.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:17:59.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.582 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:17:59.584 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:59.585+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:59.684 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:17:59.685+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:17:59.726 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.726 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:59.727 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:17:59.727 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:17:59.785 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:59.785+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:59.838 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:17:59.838 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:17:59.839 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:17:59.839 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:17:59.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:17:59.886 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:17:59.887+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:17:59.987 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:17:59.988+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.487372757000003, 2024-09-16T22:17:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:17:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:17:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485826979999999, 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:17:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:17:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485808900000002, 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:18:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:18:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485792338000003, 2024-09-16T22:18:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:18:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485775850000003, 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:18:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:18:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485758777999997, 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:18:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:18:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485742272000003, 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:18:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.485712479999997, 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:18:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481380223000002, 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:18:00.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481357961999997, 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481336968000001, 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.039 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:18:00.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481316219999997, 2024-09-16T22:18:00.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:18:00.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481295918000001, 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.048 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:00.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.052 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481275914999998, 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:18:00.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:18:00.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.057 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481255789999999, 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:18:00.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481236195000001, 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:18:00.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:18:00.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481215272999997, 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:18:00.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:18:00.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481194485000003, 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:00.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.075 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481172823999998, 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:00.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.481143734, 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:00.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:18:00.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:18:00.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.084 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:00.126 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:00.128+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:00.189 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:00.190+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:00.232 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:00.232 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:00.232 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:00.232 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:00.290 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:00.290+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:00.388 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:00.388 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:00.388 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.391 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:00.392+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:00.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:00.491 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:00.492+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:00.500 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.317685470000001, 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:18:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:18:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.267170781000004, 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:18:00.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.253844690999998, 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.514 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:18:00.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:18:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 75.198996166000001, 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:18:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225386074999999, 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:18:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225355228000002, 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:18:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225318815000001, 2024-09-16T22:18:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:18:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225292402000001, 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:18:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:18:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225267443, 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:18:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:18:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.225248587999999, 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:18:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224933407999998, 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:18:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:18:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224918032000001, 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:18:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:18:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224302945000002, 2024-09-16T22:18:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:18:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224276104000001, 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224252033999999, 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:18:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.224025892, 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:18:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:18:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.223204495000001, 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:18:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.223182635000001, 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.222795907999998, 2024-09-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:18:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.221803018999999, 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:18:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:00.597 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:00.599 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:00.600+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:00.699 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:00.700+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:00.742 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:00.742 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:00.742 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:00.742 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:00.800 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:00.801+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:00.892 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:00.893 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:00.893 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:00.893 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:00.893 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:00.900 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:00.902+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:00.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:01.001 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:01.003+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.230045820000001, 2024-09-16T22:18:01.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:18:01.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:18:01.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.230025660999999, 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:18:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:18:01.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.230006619999997, 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:18:01.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:01.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:18:01.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:18:01.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229971212999999, 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:18:01.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229952050999998, 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:18:01.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229932769000001, 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:18:01.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:18:01.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229914071000003, 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:18:01.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:18:01.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229896334999999, 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:18:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:18:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229878704999997, 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:18:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229859486000002, 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:18:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:18:01.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229842386000001, 2024-09-16T22:18:01.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:01.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:18:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.063 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.22982365, 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:18:01.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229799802999999, 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:18:01.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:18:01.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229780824999999, 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:18:01.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229761709000002, 2024-09-16T22:18:01.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:18:01.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:18:01.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229731456000003, 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.085 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:18:01.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.229710654999998, 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:18:01.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:18:01.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.204133902000002, 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:18:01.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.204116323999997, 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:01.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:18:01.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.203098760000003, 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:01.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:01.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:18:01.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:01.111 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:01.113 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:01.114+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:01.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:01.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:18:01.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:01.215 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:01.215+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:01.259 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:01.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:01.259 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:01.259 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:01.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:01.320 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:01.322+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:01.362 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:01.381 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:01.394 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:01.394 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:01.397 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:01.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:01.415 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:01.416 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:01.417+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:01.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:01.448 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:01.448 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:01.465 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:01.482 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:01.482 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:01.516 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:01.518+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:01.524 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:01.524 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:01.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:01.617 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:01.619+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:01.682 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:01.682 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:01.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:01.731 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:01.733+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:01.818 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:01.820+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:01.838 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:01.838 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:01.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:01.919 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:01.921+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:01.994 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:01.994 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:02.020 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:02.022+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:02.110 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.110 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:02.121 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:02.123+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:02.215 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:18:02.221 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:02.223+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:02.232 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.233 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:02.322 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:02.324+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:02.395 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.395 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:02.447 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:02.449+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:02.511 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.511 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:02.560 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:02.562+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:02.624 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:02.626+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:02.667 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.667 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:02.759 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:02.761+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:02.823 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.823 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.825 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:02.827+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:02.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:02.927 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:02.928+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:02.943 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:02.943 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:02.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:03.028 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:03.030+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:03.095 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.095 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:03.128 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:03.130+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:03.206 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.206 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:03.229 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:03.231+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:03.330 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:03.331+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:03.430 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:03.432+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:03.444 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.444 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:03.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:18:03.537 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:03.539+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:03.596 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.596 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:03.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:03.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:03.636 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:03.637 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:03.649 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:03.651+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:03.653 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:03.653 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:03.669 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:03.686 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:03.686 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:03.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:03.709 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:03.709 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:03.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:03.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:03.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:03.736 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:03.736 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:03.759 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:03.761+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:03.833 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:03.835+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:03.861 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:03.861 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:03.861 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:03.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:03.954 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:03.956+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.832439101000006, 2024-09-16T22:18:04.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:04.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:18:04.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:18:04.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.781924411999995, 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:18:04.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.768598322000003, 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:18:04.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:18:04.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.713749797000006, 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:18:04.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:18:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:18:04.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:04.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740139706000001, 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.039 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:18:04.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740108858999999, 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740072445999999, 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:04.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740046032999999, 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:18:04.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:18:04.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740021074000001, 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:18:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:18:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.740002219000001, 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:18:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.739687039, 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:18:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.739671662999999, 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:18:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:18:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:18:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.739056575999999, 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:18:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:18:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.739029734999999, 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.739005665000001, 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:18:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.738779523000002, 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:18:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:18:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:18:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.737958125999999, 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:18:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.737936265999998, 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:18:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.737549539, 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:18:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:18:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:18:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:18:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.736556650000001, 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:18:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:18:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.107 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:04.108 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:18:04.108 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:18:04.108 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T22:18:04.108 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.087880 2024-09-16T22:18:04.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.08787959255602706 2024-09-16T22:18:04.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:04.110 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:04.110+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:04.209 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:04.211+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:04.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:18:04.260 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:04.260 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:04.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:04.310 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:04.312+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:04.411 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:04.412+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:04.414 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:04.414 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:04.414 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:04.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:04.512 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:04.513+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791931722999998, 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791911564000003, 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:04.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:18:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:18:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791892523000001, 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:18:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:18:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:18:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791857116000003, 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:18:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:18:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791837954000002, 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:18:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:18:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791818671999998, 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791799974, 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:18:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:18:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791782238000003, 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:18:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:18:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791764608000001, 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791745388999999, 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:18:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:18:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791728288999998, 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:18:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:18:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791709552999997, 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:18:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:18:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791685706000003, 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791666728000003, 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:18:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791647611999998, 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:18:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791617359, 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:18:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.791596558000002, 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:18:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.766019804999999, 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.766002227000001, 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:18:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:18:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:18:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.764984663, 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:04.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:04.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:04.658 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:04.659+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:04.758 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:04.760+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:04.767 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:04.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:04.767 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:04.767 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:04.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:04.860 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:04.862+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:04.924 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:04.924 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:04.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:04.973 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:04.975+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.061 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:05.063+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:18:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.020232929999999, 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:18:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:18:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.432008787999997, 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:18:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695792801, 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:18:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:18:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695772377000001, 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695752370000001, 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:18:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:18:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695732086, 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695561849000001, 2024-09-16T22:18:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695531957, 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:18:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695508319999998, 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:18:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695486254999999, 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695465325000001, 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695438481, 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:18:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:18:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695417115000001, 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:18:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.69539632, 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:18:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695369612, 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695343421, 2024-09-16T22:18:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695321689, 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:18:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695300757999998, 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695280222000001, 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.695247691999999, 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:18:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.164 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:05.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:05.166 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:05.167+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.225 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 0.0878796 (857592) 2024-09-16T22:18:05.225 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:18:05.290 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:05.292+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.322 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:05.322 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:05.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:05.383 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:05.373+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.467 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:05.469+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.478 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:05.479 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:05.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:05.569 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:05.571+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:05.635 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054210480999998, 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:18:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:18:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:18:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054190738999999, 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054161289, 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:18:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054135662, 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054113009000002, 2024-09-16T22:18:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054090031000001, 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:18:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:18:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054066978000002, 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054045257999999, 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:18:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:18:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054025054, 2024-09-16T22:18:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.054006463, 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:18:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:18:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.053984932999999, 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:18:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048865837000001, 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:18:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048845568000001, 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:18:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048823437999999, 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:18:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:18:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:18:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048803957000001, 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:18:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:18:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048779617000001, 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048759651000001, 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:18:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:18:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048738239999999, 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:18:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:18:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:18:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048716478999999, 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:18:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:18:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:18:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.048696011000001, 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:18:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:18:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:05.721 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:05.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:05.723 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:05.724+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:05.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:05.823 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:05.825+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:05.831 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:05.832 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:05.832 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:05.832 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:05.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:05.891 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:05.891 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:05.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:05.909 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:05.924 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:05.926+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:05.925 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:05.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:05.942 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:05.949 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:05.949 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:05.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:05.959 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:05.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:05.977 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:06.011 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:06.064 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:06.066+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:06.066 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.067 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:06.159 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:06.161+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.224 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.226 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:06.228+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:06.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:06.327 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:06.328+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:06.387 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.388 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:06.428 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:06.430+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:06.503 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.504 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:06.529 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:06.531+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:06.628 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.629 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.632 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:06.631+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:06.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:06.730 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:06.732+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:06.741 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.741 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:06.831 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:06.833+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:06.892 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:06.892 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:06.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:06.932 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:06.933+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:07.003 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.003 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:07.055 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:07.057+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:07.115 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.115 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:07.134 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:07.136+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:07.235 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:07.237+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:07.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:18:07.273 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.273 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:07.336 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:07.337+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:07.395 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.395 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:07.437 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:07.439+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:07.538 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:07.539+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:07.550 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.550 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:07.639 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:07.641+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:07.702 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.703 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:07.740 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:07.742+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:07.841 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:07.843+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:07.923 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:07.923 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:07.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:07.942 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:07.944+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:08.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:18:08.033 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:08.034 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:08.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:08.043 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:08.045+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:08.144 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:08.146+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:08.155 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:08.172 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:08.172 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:08.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:08.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:08.179 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:08.180 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:08.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:08.189 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:08.189 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:08.205 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:08.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:08.242 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:18:08.245 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:08.247+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:08.267 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:08.285 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:08.285 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:08.331 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:08.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:08.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:08.382 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:08.384+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:08.447 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:08.449+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:08.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429362933, 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:18:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:18:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.841138790999999, 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:18:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104922803999997, 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:18:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104902379999999, 2024-09-16T22:18:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:18:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:18:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104882373000002, 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:18:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104862089000001, 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:18:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:18:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104691852000002, 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104661960000001, 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104638323000003, 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:18:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:18:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104616258, 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:18:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104595328000002, 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:18:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:18:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104568483999998, 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104547117999999, 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:18:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104526323000002, 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:18:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:18:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104499615000002, 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:18:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104473423999998, 2024-09-16T22:18:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:18:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:18:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104451691999998, 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:18:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104430761000003, 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104410225000002, 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:18:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:18:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.104377694999997, 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:08.583 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:08.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:08.584 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:08.586+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:08.685 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:08.687+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:08.732 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:08.732 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:08.732 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:08.732 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:08.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:08.786 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:08.787+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:08.847 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:08.847 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:08.848 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:08.848 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:08.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:08.897 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:08.899+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:08.987 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:08.989+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:09.005 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.005 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:09.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:09.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:09.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423768035000002, 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:18:09.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423748293000003, 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423718843000003, 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423693215999997, 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:18:09.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423670563000002, 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:18:09.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423647584999998, 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:18:09.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423624531999998, 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423602811999999, 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:18:09.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423582607999997, 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:18:09.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423564016999997, 2024-09-16T22:18:09.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:09.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.423542486999999, 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:09.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:18:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418423390999997, 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:18:09.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:18:09.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:18:09.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418403122000001, 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418380992000003, 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:18:09.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:18:09.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:18:09.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418361511000001, 2024-09-16T22:18:09.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:18:09.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418337170999997, 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:18:09.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418317205000001, 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:18:09.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418295794000002, 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:09.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:18:09.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:18:09.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:18:09.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418274033000003, 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:09.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:18:09.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.418253565000001, 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:18:09.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.121 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:09.123 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:09.124+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:09.223 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:09.225+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:09.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:09.279 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:09.279 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:09.324 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:09.326+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:09.425 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:09.427+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:09.435 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:09.436 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:09.526 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:09.527+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.977025964000006, 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:18:09.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:18:09.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.977006623999998, 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:18:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:18:09.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.976602224000004, 2024-09-16T22:18:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:09.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.974906954000005, 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:18:09.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:18:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.974876421000005, 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:18:09.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:18:09.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.970475522000001, 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:09.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:18:09.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.970455040999994, 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:18:09.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.970434464999997, 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.970409979999999, 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.970388408999995, 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:18:09.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:18:09.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964309811000007, 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:18:09.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:18:09.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964289479000001, 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:18:09.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:18:09.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964268259999997, 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:09.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:18:09.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964246888000005, 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964226049999994, 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:18:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964205469000007, 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:18:09.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.964183902000002, 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:18:09.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.959537028, 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:18:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:09.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.904231627000001, 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:09.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:18:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.227042644000001, 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:18:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:18:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:18:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:09.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:09.646 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:09.647+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:09.747 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:09.749+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:09.794 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:09.794 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:09.794 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:09.848 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:09.850+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:09.949 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:09.951+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:09.950 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:09.950 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:09.950 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:09.950 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:09.950 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:09.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:10.050 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:10.051+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 126.640554851, 2024-09-16T22:18:10.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:10.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:18:10.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:18:10.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.805749798999997, 2024-09-16T22:18:10.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:18:10.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.805728959999996, 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:18:10.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.805705720000006, 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:18:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:18:10.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.805003834999994, 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:18:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.801871387999995, 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:18:10.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:18:10.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.095 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.801461778000004, 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:18:10.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:18:10.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.801175600999997, 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:18:10.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:18:10.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.800878377000004, 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:18:10.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:18:10.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.800090134000001, 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:18:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:18:10.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.799975364999995, 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:18:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.799724147999996, 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:18:10.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:18:10.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.799415789999998, 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:18:10.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:18:10.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.799376852999998, 2024-09-16T22:18:10.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:10.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:18:10.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.799091433000001, 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.793239929999999, 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:18:10.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.793215532000005, 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:10.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:18:10.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.793164881999999, 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:10.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:18:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.793131184999993, 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:18:10.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:10.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.793020214999999, 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:10.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:10.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:10.162 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:10.163+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:10.250 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:18:10.250 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:18:10.250 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T22:18:10.250 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:18:10.250 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:18:10.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-16T22:18:10.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:18:10.294 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:10.294 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:10.295 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:10.295 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:10.296+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:10.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:10.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:10.363 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:10.364+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:10.382 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:10.382 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-16T22:18:10.382 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:10.443 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:10.444 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:10.462 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:10.463 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:10.465+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:10.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:10.479 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:10.479 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:10.512 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:10.513 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:10.564 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:10.565+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:10.610 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:10.610 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:10.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:10.664 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:10.666+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:10.722 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:10.724 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:10.724 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:10.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:10.731 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:10.731 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:10.747 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:10.748 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:10.764 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:10.765 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:10.772 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:10.774+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:10.866 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:10.868+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:10.874 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:10.874 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:10.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:10.967 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:10.969+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:10.985 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:10.985 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:10.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:11.068 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:11.070+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:11.097 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.097 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:11.169 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:11.171+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:18:11.299 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:11.301+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:11.367 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.367 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.370 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:11.372+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:11.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:11.471 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:11.473+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:11.520 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.521 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:11.572 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:11.574+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:11.672 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.672 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.673 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:11.674+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:11.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:11.773 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:11.775+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:11.873 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:11.875+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:11.912 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:11.912 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:11.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:11.974 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:11.976+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:12.030 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.030 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:12.075 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:12.076+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.145 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:12.197 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:12.199+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:12.261 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.261 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:12.276 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:12.277+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:12.377 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:12.379+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:12.428 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.428 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:12.477 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:12.479+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:12.579 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:12.580+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:12.583 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.583 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:12.680 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:12.682+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:12.740 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.740 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:12.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:18:12.780 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:12.782+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:12.882 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:12.883+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:12.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:12.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:12.915 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:12.934 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:12.934 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:12.952 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:12.952 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:12.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:12.972 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:12.982 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:12.984+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:12.986 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:12.986 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:13.003 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:13.004 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:13.018 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.442873996000003, 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:18:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.442854655999994, 2024-09-16T22:18:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:18:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.442450256000001, 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:18:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.440754986000002, 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:18:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.440724453000001, 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.436323553999998, 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:18:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.436303073000005, 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.436282496999993, 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:18:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.436258011999996, 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:18:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.436236441000005, 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430157843000003, 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:18:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430137510999998, 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430116291999994, 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:18:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430094920000002, 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430074082000004, 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:18:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430053501000003, 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:18:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.430031933999999, 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.425385059999996, 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:18:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.370079658999998, 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.692890675999998, 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:18:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.109 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:13.110 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:13.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:13.111 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:13.112 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:13.112 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:13.112+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:13.211 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:13.213+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:13.260 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:13.260 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:13.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:13.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:18:13.313 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:13.315+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:13.396 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:13.396 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:13.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:13.449 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:13.451+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:13.514 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:13.516+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:13.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 130.090959873, 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.27858260200000001, 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4217", 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/832144218", 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427490+0000", 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427491+0000", 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.238e-06 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427497+0000", 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-16T22:18:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427509+0000", 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2697999999999999e-05 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427516+0000", 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7479999999999996e-06 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427576+0000", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9948999999999998e-05 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.427583+0000", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3529999999999997e-06 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.705910+0000", 2024-09-16T22:18:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.278326988 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:03.706073+0000", 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162518 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53845 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.256154820999996, 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377743999999997, 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53845 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262295+0000", 2024-09-16T22:18:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262296+0000", 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262298+0000", 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262304+0000", 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0780000000000001e-06 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262308+0000", 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3640000000000001e-06 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338502+0000", 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076194026999999998 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338582+0000", 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9320000000000006e-05 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338673+0000", 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1333000000000001e-05 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53846 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.256133981999994, 2024-09-16T22:18:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076560353999999997, 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53846 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262316+0000", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262317+0000", 2024-09-16T22:18:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262318+0000", 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9539999999999998e-06 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262326+0000", 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262332+0000", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338703+0000", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076371622 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338779+0000", 2024-09-16T22:18:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338876+0000", 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.781e-05 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53847 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.256110742000004, 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076761456000000006, 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53847 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262339+0000", 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262842+0000", 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050284199999999998 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262847+0000", 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4710000000000001e-06 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.262860+0000", 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3033999999999999e-05 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263027+0000", 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167224 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.338908+0000", 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075881555000000003 2024-09-16T22:18:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339000+0000", 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1490000000000007e-05 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339101+0000", 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010084 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53848 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.255408857000006, 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076276113000000006, 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53848 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263041+0000", 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263438+0000", 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039682900000000002 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263443+0000", 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-16T22:18:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263455+0000", 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1559000000000001e-05 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.263463+0000", 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6449999999999994e-06 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339133+0000", 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075670050000000003 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339214+0000", 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1002999999999997e-05 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.339317+0000", 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102927 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53856 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 634880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.252276409999993, 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076344391999999997, 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53856 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266174+0000", 2024-09-16T22:18:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266544+0000", 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000370448 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266549+0000", 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1889999999999998e-06 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266562+0000", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2944e-05 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266570+0000", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6450000000000005e-06 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342048+0000", 2024-09-16T22:18:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075478300999999998 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342299+0000", 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025033999999999999 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342518+0000", 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021952500000000001 2024-09-16T22:18:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53857 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1933312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.251866800000002, 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076390265999999998, 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53857 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266583+0000", 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266833+0000", 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024935099999999998 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266838+0000", 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266849+0000", 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0913999999999999e-05 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266856+0000", 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7680000000000008e-06 2024-09-16T22:18:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342549+0000", 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075692741999999993 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342782+0000", 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023229300000000001 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.342974+0000", 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019203699999999999 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53858 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.251580622999995, 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076500410000000005, 2024-09-16T22:18:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53858 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.266869+0000", 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267129+0000", 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000259182 2024-09-16T22:18:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267133+0000", 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6890000000000003e-06 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267144+0000", 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0380000000000001e-05 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267155+0000", 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1035e-05 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343007+0000", 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075851959999999996 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343210+0000", 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203224 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343370+0000", 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015993999999999999 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53859 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.251283399000002, 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076657192999999998, 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53859 2024-09-16T22:18:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267167+0000", 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267914+0000", 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074770000000000001 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267926+0000", 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1571e-05 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267936+0000", 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0213e-05 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267943+0000", 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343404+0000", 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075460592000000007 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343596+0000", 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192468 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343824+0000", 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022755900000000001 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53860 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 667648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.250495155999999, 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076331110999999993, 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53860 2024-09-16T22:18:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.267955+0000", 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268036+0000", 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1544000000000004e-05 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268041+0000", 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268050+0000", 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1349999999999998e-06 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268057+0000", 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.343858+0000", 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075800510000000001 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344099+0000", 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241742 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344286+0000", 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018671000000000001 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53861 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.250380387000007, 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076461828999999995, 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53861 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268070+0000", 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268280+0000", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021078000000000001 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268286+0000", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268298+0000", 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2043e-05 2024-09-16T22:18:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268306+0000", 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344320+0000", 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076013171000000004 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344403+0000", 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3325000000000006e-05 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344532+0000", 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128701 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53862 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.250129169999994, 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076464114, 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53862 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268321+0000", 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268561+0000", 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023983000000000001 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268566+0000", 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268577+0000", 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-16T22:18:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268584+0000", 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344570+0000", 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075985576999999999 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344664+0000", 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4050999999999998e-05 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344785+0000", 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121426 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53863 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 684032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.249820811999996, 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076377432999999995, 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53863 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268629+0000", 2024-09-16T22:18:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268630+0000", 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268633+0000", 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-16T22:18:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268642+0000", 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-06 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268648+0000", 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2949999999999999e-06 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344818+0000", 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076169859000000006 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.344898+0000", 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0094999999999998e-05 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345007+0000", 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010885799999999999 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53864 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.249781874999996, 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076550477000000006, 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53864 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268668+0000", 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268912+0000", 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024397500000000001 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268923+0000", 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1083e-05 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268935+0000", 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1348999999999999e-05 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268942+0000", 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1720000000000001e-06 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345038+0000", 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076096300000000006 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345124+0000", 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6254999999999993e-05 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345219+0000", 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4343000000000001e-05 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53865 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 700416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.249496454999999, 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076482020999999997, 2024-09-16T22:18:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53865 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.268954+0000", 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274016+0000", 2024-09-16T22:18:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050625669999999996 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274022+0000", 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274033+0000", 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1024e-05 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274042+0000", 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3480000000000004e-06 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345247+0000", 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071205192 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345334+0000", 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7128999999999999e-05 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.345436+0000", 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101798 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53888 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1118208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.243644951999997, 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076314787999999995, 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53888 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274805+0000", 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274806+0000", 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274809+0000", 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274816+0000", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9609999999999999e-06 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274821+0000", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999997e-06 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351020+0000", 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076199320000000001 2024-09-16T22:18:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351065+0000", 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5039e-05 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351120+0000", 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4598999999999997e-05 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53889 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.243620554000003, 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076481384999999999, 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53889 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274830+0000", 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274832+0000", 2024-09-16T22:18:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274839+0000", 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8580000000000002e-06 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274845+0000", 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4389999999999999e-06 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351149+0000", 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076304219000000006 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351220+0000", 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1248999999999995e-05 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351311+0000", 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0908000000000004e-05 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53890 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.243569903999997, 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076641231000000004, 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53890 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274880+0000", 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274881+0000", 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274888+0000", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274897+0000", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.465e-06 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274904+0000", 2024-09-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.861e-06 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351342+0000", 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076437725999999998 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351424+0000", 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2303000000000001e-05 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351521+0000", 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7164999999999996e-05 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53891 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.243536207000005, 2024-09-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076824749999999997, 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53891 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274914+0000", 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274917+0000", 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2589999999999999e-06 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.274998+0000", 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275007+0000", 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4999999999999999e-06 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351554+0000", 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076547372000000002 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351643+0000", 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8512999999999998e-05 2024-09-16T22:18:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351739+0000", 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6034999999999998e-05 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:53892 3.14 3:286f1fd6:::rbd_data.11cd13540b0c.0000000000000000:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.243425236999997, 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076901311999999999, 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 53892 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.275025+0000", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280946+0000", 2024-09-16T22:18:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059210469999999996 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280956+0000", 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7559999999999994e-06 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280967+0000", 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039000000000001e-05 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.280974+0000", 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0439999999999998e-06 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351769+0000", 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070794983000000006 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351822+0000", 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3338000000000003e-05 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:58.351926+0000", 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010410499999999999 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:13.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:13.628 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:13.629+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:13.727 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:13.729+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:13.747 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:13.747 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:13.747 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:13.747 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:13.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:13.841 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:13.843+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:13.903 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:13.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:13.903 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:13.903 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:13.903 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:13.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:13.953 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:13.955+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:14.030 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:14.031+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132774 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.467126870999998, 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042286000000006, 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132774 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590841+0000", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590842+0000", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590844+0000", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1670000000000002e-06 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590850+0000", 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0820000000000004e-06 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590854+0000", 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6500000000000004e-06 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657642+0000", 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066787114999999994 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657686+0000", 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4026999999999997e-05 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657883+0000", 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019783499999999999 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132776 3.6 3:6114d16a:::rbd_data.11cd14846e35.0000000000000007:26 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.467088296999997, 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067347559000000001, 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132776 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590880+0000", 2024-09-16T22:18:14.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590882+0000", 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7659999999999999e-06 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590887+0000", 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4020000000000001e-06 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590893+0000", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8170000000000004e-06 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657916+0000", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067022679000000002 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.657989+0000", 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3372000000000002e-05 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658227+0000", 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023796000000000001 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132777 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.467068648000001, 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360191999999999, 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132777 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590899+0000", 2024-09-16T22:18:14.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590900+0000", 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590901+0000", 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.748e-06 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590908+0000", 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5359999999999998e-06 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590913+0000", 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.814e-06 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658255+0000", 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067341783000000002 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658260+0000", 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9400000000000001e-06 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132779 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.467027235000003, 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067345548000000005, 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132779 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590941+0000", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590943+0000", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.829e-06 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590949+0000", 2024-09-16T22:18:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7200000000000003e-06 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590953+0000", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.16e-06 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658280+0000", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327303000000005 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658286+0000", 2024-09-16T22:18:14.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1260000000000003e-06 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132781 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466991385, 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067337527999999994, 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132781 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590977+0000", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590979+0000", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.809e-06 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590984+0000", 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.344e-06 2024-09-16T22:18:14.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590989+0000", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658309+0000", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067320649999999996 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658314+0000", 2024-09-16T22:18:14.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8019999999999999e-06 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132782 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466973461999999, 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343799999999995, 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132782 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:18:14.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590995+0000", 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.590997+0000", 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6670000000000001e-06 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591003+0000", 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591008+0000", 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2809999999999999e-06 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658334+0000", 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067325839999999998 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658338+0000", 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4830000000000001e-06 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132783 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466953881000002, 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067348081000000004, 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132783 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591014+0000", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591015+0000", 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6899999999999998e-07 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591022+0000", 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591027+0000", 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9270000000000002e-06 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591033+0000", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3580000000000002e-06 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658358+0000", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067324720000000005 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658362+0000", 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.702e-06 2024-09-16T22:18:14.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132784 3.6 3.17510406 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466928369999998, 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067346488999999995, 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132784 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591040+0000", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.89e-07 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591042+0000", 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.874e-06 2024-09-16T22:18:14.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591048+0000", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6670000000000001e-06 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591052+0000", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.121e-06 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658382+0000", 2024-09-16T22:18:14.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067330158000000001 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658386+0000", 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2799999999999997e-06 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132785 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466909987999998, 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067354503999999996, 2024-09-16T22:18:14.091 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132785 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591058+0000", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591060+0000", 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8929999999999999e-06 2024-09-16T22:18:14.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591066+0000", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6609999999999997e-06 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591071+0000", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.972e-06 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658404+0000", 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067333031000000002 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658413+0000", 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5259999999999993e-06 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132788 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466891447999998, 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067360176999999993, 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132788 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591077+0000", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6600000000000002e-07 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591079+0000", 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.813e-06 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591085+0000", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.84e-06 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591089+0000", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4610000000000001e-06 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658432+0000", 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067343397999999999 2024-09-16T22:18:14.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658437+0000", 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2989999999999998e-06 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132790 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466873268000001, 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369473999999999, 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132790 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591095+0000", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591097+0000", 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-16T22:18:14.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591102+0000", 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5319999999999997e-06 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591107+0000", 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4089999999999996e-06 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658458+0000", 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067351564000000003 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658464+0000", 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8050000000000003e-06 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132792 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:18:14.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466855701999997, 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067394067000000002, 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132792 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591112+0000", 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591113+0000", 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591114+0000", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7680000000000001e-06 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591120+0000", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5709999999999997e-06 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591129+0000", 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5580000000000001e-06 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658497+0000", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067367983000000006 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658506+0000", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8160000000000005e-06 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132793 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:18:14.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466833739999998, 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067386094999999993, 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132793 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:18:14.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591134+0000", 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591135+0000", 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591136+0000", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6950000000000001e-06 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591142+0000", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8699999999999997e-06 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591146+0000", 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658516+0000", 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067369831000000005 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658520+0000", 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2690000000000001e-06 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132795 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466815816, 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409281000000001, 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132795 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591152+0000", 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591153+0000", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9499999999999998e-07 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591154+0000", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7990000000000001e-06 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591160+0000", 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6330000000000001e-06 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591164+0000", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658549+0000", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067385006999999997 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658561+0000", 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2062000000000001e-05 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.115 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132796 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466798306000001, 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067427494000000004, 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132796 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591170+0000", 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2300000000000002e-07 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591172+0000", 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8500000000000001e-06 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591178+0000", 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7490000000000004e-06 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591182+0000", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1790000000000001e-06 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658591+0000", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067409580999999996 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658597+0000", 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8119999999999996e-06 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132798 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466780634999999, 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067448739999999993, 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132798 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591187+0000", 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591596+0000", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040859300000000001 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591599+0000", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5159999999999999e-06 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591612+0000", 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2692e-05 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591618+0000", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9859999999999999e-06 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658628+0000", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067009465000000004 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658636+0000", 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4880000000000007e-06 2024-09-16T22:18:14.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132799 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466338694000001, 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042347000000002, 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132799 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591629+0000", 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591630+0000", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591632+0000", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.464e-06 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591640+0000", 2024-09-16T22:18:14.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9349999999999994e-06 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591645+0000", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5700000000000003e-06 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658661+0000", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067016266000000005 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658672+0000", 2024-09-16T22:18:14.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0631e-05 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132801 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466223397, 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953950999999998, 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132801 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:18:14.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591745+0000", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591748+0000", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7609999999999999e-06 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591757+0000", 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-16T22:18:14.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591763+0000", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.105e-06 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658691+0000", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927826999999995 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658699+0000", 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9789999999999993e-06 2024-09-16T22:18:14.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132803 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466196398000001, 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066957056000000001, 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.132 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132803 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591772+0000", 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591779+0000", 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4710000000000004e-06 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591791+0000", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2711000000000001e-05 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591797+0000", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5740000000000003e-06 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658724+0000", 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066927367000000001 2024-09-16T22:18:14.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658729+0000", 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4680000000000003e-06 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:132806 3.6 3.568b2886 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.466143021000001, 2024-09-16T22:18:14.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066953258000000002, 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 132806 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.591825+0000", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592299+0000", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047417800000000001 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592304+0000", 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8910000000000003e-06 2024-09-16T22:18:14.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592323+0000", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8906000000000002e-05 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.592334+0000", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012000000000001e-05 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658773+0000", 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066439023999999999 2024-09-16T22:18:14.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:18.658778+0000", 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2469999999999999e-06 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.140 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.141 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:18:14.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:14.142 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:14.143+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:14.242 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:14.243+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:14.291 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.291 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:14.292 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:14.292 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:14.344 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:14.346+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:14.404 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.404 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:14.404 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:14.404 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:14.404 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:14.443 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:14.445+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:14.544 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:14.546+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:14.556 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.052083426999999, 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:18:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:18:14.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050537649999995, 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:18:14.561 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050519570000006, 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:18:14.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050503008000007, 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:18:14.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:14.574 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050486520000007, 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:18:14.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:18:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050469448000001, 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.050452942000007, 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:18:14.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.05042315, 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.046090892999999, 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.046068632000001, 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.046047638000005, 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.046026889999993, 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:14.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.046006587999997, 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:18:14.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045986584999994, 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:14.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:14.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045966460000002, 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:14.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045946865000005, 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:18:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:18:14.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:14.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045925943, 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:14.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045905155, 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:18:14.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045883493999995, 2024-09-16T22:18:14.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:14.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:18:14.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.045854403999996, 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:18:14.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:18:14.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.645 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:14.647 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:14.648+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:14.747 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:14.749+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:14.749 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:14.749 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:14.848 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:14.850+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:14.859 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.859 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:14.859 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:14.859 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:14.859 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:14.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:14.950 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:14.952+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.78978199700001, 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:18:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:18:14.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:18:14.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:18:14.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:18:14.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.739267308, 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.980 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:18:14.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:14.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 108.725941218, 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:18:14.985 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:18:14.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:18:14.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.671092693000006, 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:18:14.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:18:14.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:18:14.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:18:14.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:14.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697482602000001, 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:14.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:18:15.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697451755000003, 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:18:15.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697415341999999, 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697388928999999, 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697363969999998, 2024-09-16T22:18:15.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:18:15.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:18:15.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.018 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697345114999997, 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:18:15.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697029935000003, 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.697014559000003, 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.696399472000003, 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:18:15.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:18:15.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.696372631000003, 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.696348561000001, 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:18:15.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.696122418999998, 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.695301022000002, 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:18:15.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:18:15.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.695279161999999, 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:18:15.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:18:15.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:18:15.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.694892435, 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.693899545999997, 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:15.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:15.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:15.066 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:15.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:15.068 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:15.069+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:15.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:15.168 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:15.170+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:15.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:15.269 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:15.271+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:15.270 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:15.270 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:15.287 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:15.288 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:15.304 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:15.304 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:15.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:15.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:15.337 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:15.370 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:15.371+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:15.370 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:15.370 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:15.373 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:15.373 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:15.373 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:15.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:15.386 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:15.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:18:15.396 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:18:15.396 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T22:18:15.396 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:18:15.396 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:18:15.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T22:18:15.480 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:15.482+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:15.484 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:15.484 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:15.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:15.544 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "" 2024-09-16T22:18:15.544 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:15.571 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:15.572+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:15.601 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:15.601 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:15.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:15.672 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:15.674+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:15.757 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:15.757 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:15.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:15.773 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:15.775+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:15.873 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:15.873 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:15.874 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:15.875+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:15.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:15.974 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:15.976+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:16.030 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.030 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:16.075 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:16.077+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:16.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T22:18:16.193 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:16.195+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:16.202 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.202 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:16.276 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:16.278+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:16.363 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.363 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:16.415 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:16.417+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:16.479 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:16.481+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:16.479 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.479 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:16.579 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:16.581+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:16.636 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.637 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:16.685 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:16.687+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:16.781 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:16.783+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:16.882 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:16.884+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:16.907 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:16.908 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:16.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:16.983 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:16.984+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:17.063 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:17.063 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:17.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:17.115 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:17.117+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-16T22:18:17.184 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:17.186+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:17.189 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:17.189 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:17.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:17.285 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:17.287+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:17.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:17.386 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:17.388+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:17.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:18:17.426 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:17.426 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:17.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:17.487 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:17.489+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:17.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:17.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:17.540 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:17.540 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:17.556 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:17.556 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:17.572 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:17.578 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:17.578 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:17.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:17.588 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:17.589+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:17.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:17.606 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:17.606 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:17.623 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:17.623 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:17.640 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:17.640 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:17.692 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:17.694+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:17.732 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:17.732 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:17.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:17.793 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:17.794+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:17.842 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:17.843 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:17.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:17.893 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:17.895+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:17.994 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:17.996+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:17.996 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:17.996 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:17.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77071 3.0 3:01338d7e:::rbd_data.11cd4ce083ff.0000000000000041:head [stat,delete] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.491881235999998, 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056138030999999998, 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77071 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505896+0000", 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505897+0000", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9199999999999999e-07 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505902+0000", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7979999999999996e-06 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505920+0000", 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7867e-05 2024-09-16T22:18:17.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.505927+0000", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0929999999999997e-06 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557848+0000", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05192099 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.557942+0000", 2024-09-16T22:18:18.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4062999999999997e-05 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562034+0000", 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040922279999999998 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77140 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490335458999994, 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054127386, 2024-09-16T22:18:18.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77140 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507442+0000", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507444+0000", 2024-09-16T22:18:18.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507450+0000", 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5550000000000002e-06 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507455+0000", 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0490000000000003e-06 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561317+0000", 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053862045999999997 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561419+0000", 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102305 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561569+0000", 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150013 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77141 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490317379000004, 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054343897000000002, 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77141 2024-09-16T22:18:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507460+0000", 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3700000000000001e-07 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507462+0000", 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7320000000000001e-06 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507467+0000", 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3469999999999998e-06 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507472+0000", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2320000000000003e-06 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561611+0000", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054139593999999999 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561714+0000", 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102957 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.561804+0000", 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9697999999999994e-05 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77142 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490300817000005, 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056130416000000002, 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77142 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507477+0000", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8099999999999999e-07 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507478+0000", 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-16T22:18:18.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507484+0000", 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3460000000000001e-06 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507488+0000", 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8940000000000003e-06 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.562889+0000", 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055401044000000003 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563002+0000", 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011322799999999999 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563607+0000", 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060502600000000004 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77143 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490284329000005, 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458622, 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77143 2024-09-16T22:18:18.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507493+0000", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2099999999999998e-07 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507495+0000", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.751e-06 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507501+0000", 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-16T22:18:18.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507505+0000", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563639+0000", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056134556000000002 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563743+0000", 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010343 2024-09-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563952+0000", 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208939 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77144 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3428352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490267256999999, 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056763582999999999, 2024-09-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77144 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507510+0000", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.46e-07 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507512+0000", 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2024-09-16T22:18:18.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507517+0000", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3720000000000004e-06 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507522+0000", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2980000000000002e-06 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.563983+0000", 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056461566999999997 2024-09-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564113+0000", 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012975000000000001 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564274+0000", 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160672 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77145 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490250751000005, 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057080068999999997, 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77145 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507527+0000", 2024-09-16T22:18:18.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507539+0000", 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2490000000000001e-05 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507546+0000", 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.127e-06 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507550+0000", 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7010000000000004e-06 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564304+0000", 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056753493000000002 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564406+0000", 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102154 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564607+0000", 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020074399999999999 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77146 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.490220958999998, 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057383825999999999, 2024-09-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77146 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507556+0000", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507557+0000", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507559+0000", 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507564+0000", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.429e-06 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.507569+0000", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8319999999999996e-06 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564660+0000", 2024-09-16T22:18:18.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057090928999999999 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564786+0000", 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126649 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.564940+0000", 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153843 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77260 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485888701999997, 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054070462, 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77260 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511889+0000", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511891+0000", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511899+0000", 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2219999999999996e-06 2024-09-16T22:18:18.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511904+0000", 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-06 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565653+0000", 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053749287 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565774+0000", 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121131 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565959+0000", 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185069 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77261 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:18.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485866440999999, 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054465424999999998, 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77261 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511911+0000", 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4099999999999999e-07 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511914+0000", 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2050000000000001e-06 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511920+0000", 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5690000000000001e-06 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511925+0000", 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.565991+0000", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054066220999999998 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566108+0000", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011618200000000001 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566376+0000", 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026883300000000001 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77262 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3862528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485845447000003, 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054770215999999997, 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77262 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511932+0000", 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511934+0000", 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511941+0000", 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.63e-06 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511946+0000", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0359999999999997e-06 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566468+0000", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054522159000000001 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566575+0000", 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106847 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566702+0000", 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012710299999999999 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77263 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485824699000005, 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054969489000000003, 2024-09-16T22:18:18.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77263 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511953+0000", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511955+0000", 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9709999999999998e-06 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511962+0000", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5919999999999997e-06 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511967+0000", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8620000000000002e-06 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566729+0000", 2024-09-16T22:18:18.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054762890000000002 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566784+0000", 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4999e-05 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.566922+0000", 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137689 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77264 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3891200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485804396999995, 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055458398999999999, 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77264 2024-09-16T22:18:18.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511973+0000", 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511975+0000", 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.069e-06 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511982+0000", 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511987+0000", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9289999999999997e-06 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567203+0000", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055216092000000001 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567317+0000", 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113639 2024-09-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567431+0000", 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114855 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77265 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485784394000007, 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056006383, 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77265 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511993+0000", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.511995+0000", 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512002+0000", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512007+0000", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567762+0000", 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055755335000000003 2024-09-16T22:18:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567884+0000", 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121734 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.567999+0000", 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115628 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77266 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485764269000001, 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056218288999999998, 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77266 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512013+0000", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512014+0000", 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512015+0000", 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512022+0000", 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.195e-06 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512026+0000", 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568052+0000", 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056025692000000002 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568128+0000", 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6213999999999999e-05 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568231+0000", 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103057 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77267 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3960832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485744674000003, 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056458505999999999, 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77267 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512033+0000", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7899999999999999e-07 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512035+0000", 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-16T22:18:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512042+0000", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5019999999999998e-06 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512047+0000", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3879999999999999e-06 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568266+0000", 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056219445 2024-09-16T22:18:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568328+0000", 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1241000000000005e-05 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568491+0000", 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016354100000000001 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77268 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485723751999998, 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056738096000000002, 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77268 2024-09-16T22:18:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512054+0000", 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512056+0000", 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1509999999999998e-06 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512063+0000", 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5590000000000002e-06 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512068+0000", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568534+0000", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056466645000000003 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568657+0000", 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012234999999999999 2024-09-16T22:18:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568792+0000", 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013509399999999999 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77269 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4009984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485702963999998, 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056945691999999999, 2024-09-16T22:18:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77269 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512074+0000", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512075+0000", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1800000000000001e-07 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512077+0000", 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0140000000000001e-06 2024-09-16T22:18:18.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512083+0000", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4409999999999998e-06 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512088+0000", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0250000000000002e-06 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568836+0000", 2024-09-16T22:18:18.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056747517999999997 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.568915+0000", 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9451999999999998e-05 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569020+0000", 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104824 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77270 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485681303000007, 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057215805000000002, 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77270 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512096+0000", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512097+0000", 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512105+0000", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5979999999999997e-06 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512112+0000", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7209999999999997e-06 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512117+0000", 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1839999999999998e-06 2024-09-16T22:18:18.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569061+0000", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056943590000000002 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569142+0000", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1524000000000001e-05 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569312+0000", 2024-09-16T22:18:18.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169715 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:77271 3.0 3:037eef19:::rbd_data.11cdd61daadb.0000000000000004:20 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.485652212999995, 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057358065, 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.082 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77271 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512125+0000", 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512126+0000", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512128+0000", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.99e-06 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512137+0000", 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5820000000000008e-06 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.512142+0000", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569349+0000", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057206766999999999 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569408+0000", 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8430000000000001e-05 2024-09-16T22:18:18.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.569483+0000", 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5710000000000005e-05 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.086 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:18.128 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:18.130+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:18.197 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:18.199+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:18.240 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4557.0:218895 3.b e130/23 3:d7f4931b:::rbd_header.11cdc6143ee9:head v 130'478, mlcod=130'476)", 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:18:18.242723+0000", 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00027775100000000002, 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029742400000000002, 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242723+0000", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242723+0000", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242724+0000", 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3900000000000004e-07 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242729+0000", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6859999999999997e-06 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242740+0000", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1178e-05 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242746+0000", 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1179999999999997e-06 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242792+0000", 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5707999999999997e-05 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:18:18.242812+0000", 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0242e-05 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-16T22:18:18.244 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-16T22:18:18.245 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:18.334 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:18.336+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:18.393 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:18.394 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:18.394 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:18.394 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:18.394 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.398 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:18.400+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:18.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:18.499 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:18.501+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13401 3.1 3:870524aa:::rbd_data.11cdc482f8dc.0000000000000197:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 112.364229051, 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2523917689999999, 2024-09-16T22:18:18.547 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13401 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183688+0000", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183689+0000", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183694+0000", 2024-09-16T22:18:18.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4780000000000002e-06 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183703+0000", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8109999999999997e-06 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183708+0000", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0180000000000001e-06 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183753+0000", 2024-09-16T22:18:18.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4431000000000002e-05 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.183757+0000", 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.963e-06 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186044+0000", 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022869040000000002 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.186053+0000", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8300000000000008e-06 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434576+0000", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2485229769999999 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434703+0000", 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126663 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.434852+0000", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148906 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435966+0000", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001113689 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435976+0000", 2024-09-16T22:18:18.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.075e-05 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436010+0000", 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3269999999999998e-05 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.436080+0000", 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9945999999999997e-05 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13516 3.1 3:83703cfd:::rbd_data.11cdc482f8dc.00000000000001d6:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 112.313714362, 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.204229352, 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13516 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234203+0000", 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-07 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234207+0000", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.98e-06 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234216+0000", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4619999999999996e-06 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234221+0000", 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7439999999999998e-06 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234277+0000", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6725000000000003e-05 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.234287+0000", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8309999999999996e-06 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435260+0000", 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2009723910000001 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435373+0000", 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011358600000000001 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438432+0000", 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030587700000000002 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:13551 3.1 3:84dccac6:::rbd_data.11cdc482f8dc.00000000000001e7:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 112.30038827200001, 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1912764229999999, 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 13551 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247529+0000", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247530+0000", 2024-09-16T22:18:18.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1400000000000001e-06 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247535+0000", 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.87e-06 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247549+0000", 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4586999999999999e-05 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247557+0000", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2840000000000001e-06 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247616+0000", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9605000000000003e-05 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:26.247625+0000", 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1589999999999999e-06 2024-09-16T22:18:18.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435628+0000", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1880027870000001 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.435748+0000", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119666 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:27.438805+0000", 2024-09-16T22:18:18.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030573250000000001 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:36636 3.1 3:8066426d:::rbd_object_map.11cd43ed039f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.245539746999995, 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.98189034900000005, 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36636 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302377+0000", 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302378+0000", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2699999999999998e-07 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302382+0000", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4360000000000004e-06 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302393+0000", 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0220999999999999e-05 2024-09-16T22:18:18.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302400+0000", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302448+0000", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8829000000000003e-05 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:45.302455+0000", 2024-09-16T22:18:18.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6900000000000003e-06 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282406+0000", 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.979950824 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282516+0000", 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109967 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.282672+0000", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156534 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.283288+0000", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061516100000000003 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284166+0000", 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087843999999999997 2024-09-16T22:18:18.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284196+0000", 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9862e-05 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:16:46.284268+0000", 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1761000000000006e-05 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205629 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271929655999998, 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05232067, 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205629 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.572 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275987+0000", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275988+0000", 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9599999999999999e-07 2024-09-16T22:18:18.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275991+0000", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9840000000000001e-06 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.275999+0000", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276008+0000", 2024-09-16T22:18:18.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.327413+0000", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051404983000000001 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328239+0000", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082611100000000003 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328308+0000", 2024-09-16T22:18:18.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8597999999999996e-05 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205630 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271898809, 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052500308000000002, 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.576 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205630 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276018+0000", 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276019+0000", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276021+0000", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3039999999999999e-06 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276027+0000", 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276034+0000", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328373+0000", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052339569000000002 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328451+0000", 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.771e-05 2024-09-16T22:18:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328518+0000", 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7577999999999996e-05 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205631 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271862396000003, 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052637823, 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205631 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276055+0000", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276057+0000", 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3609999999999999e-06 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276068+0000", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0657e-05 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276073+0000", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1510000000000003e-06 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328549+0000", 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475449 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328619+0000", 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0488999999999995e-05 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328692+0000", 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3277999999999999e-05 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205632 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271835983000003, 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052802379000000003, 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205632 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276081+0000", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6199999999999998e-07 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276084+0000", 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276089+0000", 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8830000000000003e-06 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276098+0000", 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6009999999999995e-06 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328721+0000", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052622569000000001 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328817+0000", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5917999999999999e-05 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328883+0000", 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6837999999999999e-05 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205633 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271811024000002, 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052935357000000002, 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205633 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276106+0000", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276108+0000", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0480000000000001e-06 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276114+0000", 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6529999999999999e-06 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276119+0000", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6820000000000002e-06 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328911+0000", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052792036000000001 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.328980+0000", 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8880000000000005e-05 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329041+0000", 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1644999999999997e-05 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205634 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1306624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271792169000001, 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05307336, 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205634 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276125+0000", 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276410+0000", 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028549000000000001 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276415+0000", 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276422+0000", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3969999999999997e-06 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276428+0000", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329068+0000", 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052640659999999999 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329138+0000", 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9529000000000005e-05 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329198+0000", 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0167999999999997e-05 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205638 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1382400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271476989, 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052929295000000001, 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205638 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276440+0000", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276441+0000", 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.99e-07 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276443+0000", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.249e-06 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276447+0000", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276451+0000", 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1559999999999999e-06 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329227+0000", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052776242000000001 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329301+0000", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4167999999999999e-05 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329369+0000", 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8227999999999998e-05 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205640 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1404928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.271461613, 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053081534, 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205640 2024-09-16T22:18:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.276455+0000", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277022+0000", 2024-09-16T22:18:18.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056685399999999999 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277031+0000", 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7919999999999998e-06 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277050+0000", 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8907999999999999e-05 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277058+0000", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6350000000000006e-06 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329396+0000", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052338559 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329470+0000", 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4216999999999998e-05 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329537+0000", 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6569000000000006e-05 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205641 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1427456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.270846526, 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052625971000000001, 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205641 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277070+0000", 2024-09-16T22:18:18.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277071+0000", 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277074+0000", 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8049999999999998e-06 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277083+0000", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0720000000000006e-06 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277089+0000", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5690000000000004e-06 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329564+0000", 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052475215999999998 2024-09-16T22:18:18.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329633+0000", 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9653000000000005e-05 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329696+0000", 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3020000000000003e-05 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205642 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1263616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.270819684999999, 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052767008999999997, 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205642 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277097+0000", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277098+0000", 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277100+0000", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3590000000000002e-06 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277108+0000", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8720000000000002e-06 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277114+0000", 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6439999999999997e-06 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329722+0000", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052608476000000001 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329791+0000", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8429999999999994e-05 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329864+0000", 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3727999999999996e-05 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205643 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.270795614999997, 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052946160999999999, 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205643 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277121+0000", 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277315+0000", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019355500000000001 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277319+0000", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4560000000000002e-06 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277328+0000", 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4679999999999992e-06 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277334+0000", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6200000000000001e-06 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329907+0000", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052572774000000003 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.329986+0000", 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8270999999999997e-05 2024-09-16T22:18:18.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330068+0000", 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2016999999999998e-05 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205644 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1308672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.270569473000002, 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052890513, 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205644 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.277348+0000", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278130+0000", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078285799999999999 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278135+0000", 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.899e-06 2024-09-16T22:18:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278149+0000", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3901e-05 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278156+0000", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1609999999999997e-06 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330097+0000", 2024-09-16T22:18:18.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051940310000000003 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330168+0000", 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709000000000001e-05 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330238+0000", 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9675e-05 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205650 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1384448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.269748075999999, 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052245566, 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205650 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278169+0000", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278170+0000", 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-07 2024-09-16T22:18:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278172+0000", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.525e-06 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278179+0000", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9789999999999996e-06 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278184+0000", 2024-09-16T22:18:18.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0030000000000002e-06 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330268+0000", 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052083773 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330342+0000", 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3923000000000004e-05 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330414+0000", 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2749000000000004e-05 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205652 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.269726216000002, 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052378890999999997, 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205652 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278191+0000", 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278536+0000", 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000345039 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278540+0000", 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-06 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278553+0000", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3091999999999999e-05 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278565+0000", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2148e-05 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330445+0000", 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051879657000000003 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330509+0000", 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4329e-05 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330570+0000", 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0566000000000003e-05 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205653 3.4 3:26d8aa66:::rbd_data.11cdd6518f4.0000000000000051:2f [read 12288~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.269339488999996, 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05217579, 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205653 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.278577+0000", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279315+0000", 2024-09-16T22:18:18.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073706300000000004 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279319+0000", 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9060000000000001e-06 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279328+0000", 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0409999999999998e-06 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279334+0000", 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4790000000000001e-06 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330602+0000", 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05126754 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330669+0000", 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7588000000000004e-05 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.330753+0000", 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4172999999999995e-05 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:205666 3.4 3:21f572aa:::rbd_data.11cd60092c51.0000000000000007:2e [read 1386496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.268346600000001, 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052119575000000001, 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:18.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 205666 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:18.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279570+0000", 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279878+0000", 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030738899999999998 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279882+0000", 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.827e-06 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279889+0000", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1010000000000003e-06 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.279893+0000", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5249999999999999e-06 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331562+0000", 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051669092 2024-09-16T22:18:18.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331628+0000", 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5537999999999995e-05 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:36.331690+0000", 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2102999999999998e-05 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:18.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:18.642 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:18.643 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:18.644+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:18.744 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:18.746+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:18.787 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:18.788 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:18.788 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:18.788 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:18.880 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:18.882+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:18.939 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:18.939 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:18.940 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:18.946 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:18.947+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:18.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:19.047 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:19.049+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:19.092 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317519790000006, 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:18:19.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:18:19.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:19.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317499631000004, 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:18:19.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317480590000002, 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:18:19.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317445183000004, 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:18:19.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317426021000003, 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:18:19.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317406739000006, 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:18:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:18:19.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317388041000001, 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317370304999997, 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:18:19.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317352674999995, 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:19.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.133 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317333456, 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:18:19.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:18:19.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317316356000006, 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317297620000005, 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:18:19.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:18:19.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317273772999997, 2024-09-16T22:18:19.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:18:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317254794999997, 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:18:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:18:19.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317235679000007, 2024-09-16T22:18:19.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:18:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:18:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317205426000001, 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.317184624999996, 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:18:19.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.291607872, 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:18:19.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:18:19.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.291590294000002, 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:19.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.290572729999994, 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:19.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:18:19.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:19.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:19.184 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:19.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:18:19.185 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:19.187+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:19.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:19.286 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:19.288+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:19.305 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:19.305 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:19.306 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:19.306 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:19.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:19.388 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:19.390+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:19.463 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:19.463 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:19.464 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:19.464 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:19.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:19.516 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:19.518+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:19.581 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 67.520116537000007, 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:18:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:18:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:18:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.931892394999998, 2024-09-16T22:18:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:18:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:18:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195676407999997, 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195655983999998, 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195635977000002, 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195615693000001, 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:18:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195445456000002, 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:18:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:18:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195415564000001, 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:18:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:18:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195391927000003, 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:18:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195369862, 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:18:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195348932000002, 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:18:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195322087999998, 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:18:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195300721999999, 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:18:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:18:19.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195279927000001, 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:18:19.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:18:19.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195253219000001, 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:19.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195227027999998, 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195205295999997, 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:18:19.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:18:19.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:18:19.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195184365000003, 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:18:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:18:19.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195163829000002, 2024-09-16T22:18:19.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:18:19.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.195131299000003, 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:18:19.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:19.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:19.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:19.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:19.669 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:19.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:19.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:19.671 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:19.672+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:19.772 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:19.774+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:19.821 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:19.821 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:19.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:19.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:19.837 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:19.855 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:19.872 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:19.874+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:19.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:19.873 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:19.889 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:19.907 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:19.925 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:19.925 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:19.939 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:19.939 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:19.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:19.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:19.973 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:19.976+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:20.066 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.066 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:20.075 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:20.076+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:20.165 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:18:20.176 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:20.177+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:20.284 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:20.286+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:20.352 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.353 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:20.401 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:20.403+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:20.477 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:20.479+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:20.508 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.508 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:20.553 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T22:18:20.553 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T22:18:20.553 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T22:18:20.553 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T22:18:20.553 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T22:18:20.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T22:18:20.603 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:20.605+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:20.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:20.667 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "" 2024-09-16T22:18:20.668 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:20.717 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-16T22:18:20.719+0000 7f2870829640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33607) UID: 0 2024-09-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.779 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:20.781+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:20.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:20.880 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:20.882+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:20.931 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:20.932 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:20.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:20.981 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:20.983+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:21.062 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.062 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:21.119 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:21.121+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:21.184 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:21.185+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:21.198 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.198 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:21.285 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:21.286+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:21.354 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.354 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:21.386 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:21.388+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.466 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:21.518 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:21.520+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:21.577 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.577 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:21.627 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:21.629+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:21.688 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:21.690+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:21.729 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.729 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:21.789 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:21.791+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:21.881 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:21.881 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:21.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:21.890 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:21.891+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:21.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T22:18:21.990 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:21.992+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:22.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:22.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:22.126 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:22.127 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:22.129+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:22.159 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:22.177 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:22.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:18:22.190 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:22.190 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:22.197 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:22.197 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:22.198 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:22.199+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:22.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:22.217 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:22.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:22.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:22.296 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:22.298+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:22.347 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:22.347 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:22.347 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-16T22:18:22.347 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:22.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:22.401 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:22.402+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:22.458 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-16T22:18:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-16T22:18:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-16T22:18:22.459 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:22.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:22.508 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:22.510+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:22.595 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:22.596+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83245 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837712310000001, 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044174642, 2024-09-16T22:18:22.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83245 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776183+0000", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776184+0000", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776186+0000", 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8300000000000001e-06 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776192+0000", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2910000000000004e-06 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776197+0000", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820214+0000", 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044016539 2024-09-16T22:18:22.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820293+0000", 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9457999999999997e-05 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820358+0000", 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5104999999999994e-05 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83246 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6459392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837692150999999, 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044339673000000003, 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83246 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776204+0000", 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-16T22:18:22.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776206+0000", 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.216e-06 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776213+0000", 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776217+0000", 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2350000000000001e-06 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820396+0000", 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044178732999999998 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820472+0000", 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820543+0000", 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1632999999999997e-05 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83247 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6615040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837673109999997, 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044495462, 2024-09-16T22:18:22.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83247 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776223+0000", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7800000000000001e-07 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776229+0000", 2024-09-16T22:18:22.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0850000000000002e-06 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776237+0000", 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9869999999999999e-06 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776242+0000", 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3320000000000002e-06 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820575+0000", 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044333577999999998 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820652+0000", 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7228e-05 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820718+0000", 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5673999999999995e-05 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83248 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837637702999999, 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044628187, 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83248 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776258+0000", 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776259+0000", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776261+0000", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3479999999999998e-06 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776267+0000", 2024-09-16T22:18:22.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1449999999999996e-06 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776271+0000", 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2130000000000002e-06 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820747+0000", 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044475207000000003 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820819+0000", 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2510999999999997e-05 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820886+0000", 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7225999999999996e-05 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83249 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837618540999998, 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044778437999999997, 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83249 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776277+0000", 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776278+0000", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5700000000000002e-07 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776280+0000", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776286+0000", 2024-09-16T22:18:22.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3670000000000002e-06 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776290+0000", 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8979999999999998e-06 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820918+0000", 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044627748000000002 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.820993+0000", 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5056999999999996e-05 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821056+0000", 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2705000000000004e-05 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83250 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6623232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837599259000001, 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927626999999998, 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83250 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776297+0000", 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-16T22:18:22.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776299+0000", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2749999999999998e-06 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776306+0000", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3339999999999998e-06 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776310+0000", 2024-09-16T22:18:22.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9779999999999999e-06 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821083+0000", 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044773688999999998 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821149+0000", 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5586000000000005e-05 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821224+0000", 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5253000000000006e-05 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83251 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6508544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837580560999996, 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045064752999999999, 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83251 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776315+0000", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776316+0000", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9699999999999996e-07 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776318+0000", 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.092e-06 2024-09-16T22:18:22.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776324+0000", 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8300000000000001e-06 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776327+0000", 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6990000000000001e-06 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821255+0000", 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044927924000000001 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821323+0000", 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7521000000000006e-05 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821380+0000", 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7089999999999997e-05 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.645 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83252 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837562825000006, 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045184861999999999, 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83252 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776333+0000", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-07 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776335+0000", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.012e-06 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776342+0000", 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2169999999999999e-06 2024-09-16T22:18:22.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776346+0000", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.878e-06 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821409+0000", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045063179000000002 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821478+0000", 2024-09-16T22:18:22.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9565000000000002e-05 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821518+0000", 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9508999999999997e-05 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83253 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6529024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837545195000004, 2024-09-16T22:18:22.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045329814000000003, 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83253 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776351+0000", 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1099999999999996e-07 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776353+0000", 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776359+0000", 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9370000000000001e-06 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776364+0000", 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821546+0000", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045181820999999997 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821617+0000", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0852999999999994e-05 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821680+0000", 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3876999999999998e-05 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83254 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6664192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837525975999995, 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045451317999999997, 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.655 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83254 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776370+0000", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776372+0000", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776377+0000", 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8810000000000004e-06 2024-09-16T22:18:22.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776382+0000", 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5199999999999999e-06 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821705+0000", 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045323640999999998 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821758+0000", 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2584999999999999e-05 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821821+0000", 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3101000000000005e-05 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83255 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6549504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:22.659 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837508876000001, 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045602581000000003, 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83255 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776387+0000", 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776390+0000", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2290000000000002e-06 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776397+0000", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9920000000000001e-06 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776401+0000", 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1869999999999999e-06 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821855+0000", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045453651999999997 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821930+0000", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5518000000000004e-05 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.821990+0000", 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.948e-05 2024-09-16T22:18:22.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83257 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6557696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.83749014, 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045756092999999998, 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.664 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83257 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776406+0000", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7700000000000004e-07 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776409+0000", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1900000000000002e-06 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776419+0000", 2024-09-16T22:18:22.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0907000000000001e-05 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776424+0000", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.104e-06 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822017+0000", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045593316000000002 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822097+0000", 2024-09-16T22:18:22.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9824e-05 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822162+0000", 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5074999999999996e-05 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83259 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6578176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:22.668 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837466293000006, 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045867272000000001, 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83259 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:22.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776430+0000", 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776432+0000", 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.052e-06 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776438+0000", 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776442+0000", 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7850000000000002e-06 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822182+0000", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045740201000000001 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822238+0000", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6116000000000003e-05 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822297+0000", 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8913000000000003e-05 2024-09-16T22:18:22.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83261 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6598656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837447315000006, 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045988326000000003, 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83261 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776448+0000", 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776449+0000", 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776451+0000", 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1110000000000002e-06 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776458+0000", 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7680000000000003e-06 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776462+0000", 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9990000000000002e-06 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822327+0000", 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045864755 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822398+0000", 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0976999999999995e-05 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822437+0000", 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9168000000000002e-05 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83264 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6627328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837428199000001, 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046071708000000003, 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83264 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.678 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776468+0000", 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776477+0000", 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0189999999999995e-06 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776487+0000", 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8670000000000006e-06 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776492+0000", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5970000000000002e-06 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822461+0000", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045969430999999998 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822504+0000", 2024-09-16T22:18:22.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822539+0000", 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5669e-05 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83265 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6647808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:22.682 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837397945999996, 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046228017000000003, 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83265 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:22.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776498+0000", 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776500+0000", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.905e-06 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776508+0000", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776512+0000", 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9030000000000003e-06 2024-09-16T22:18:22.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822569+0000", 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046056765 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822646+0000", 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6866999999999994e-05 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822726+0000", 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9822999999999998e-05 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83267 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.837377145000005, 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046389263999999999, 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83267 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776519+0000", 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776521+0000", 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776527+0000", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.776533+0000", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2090000000000002e-06 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822755+0000", 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046221579999999998 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822839+0000", 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3475999999999998e-05 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.822908+0000", 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9408999999999999e-05 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83942 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.811800391999995, 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044078160999999998, 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83942 2024-09-16T22:18:22.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802095+0000", 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802096+0000", 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4999999999999998e-07 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802098+0000", 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.776e-06 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802103+0000", 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2900000000000002e-06 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802107+0000", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.403e-06 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846035+0000", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043927900999999998 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846112+0000", 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-16T22:18:22.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846174+0000", 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2003999999999998e-05 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83943 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6803456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.811782813999997, 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044231076000000001, 2024-09-16T22:18:22.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83943 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802113+0000", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802115+0000", 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.671e-06 2024-09-16T22:18:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802120+0000", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.361e-06 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.802126+0000", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8080000000000001e-06 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846202+0000", 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044075989000000003 2024-09-16T22:18:22.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846276+0000", 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3823000000000001e-05 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.846344+0000", 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8056e-05 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4557.0:83948 3.b 3:d5a01792:::rbd_data.11cdd61daadb.0000000000000006:20 [read 6901760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.810765250000003, 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044063098000000002, 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4557", 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 83948 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-16T22:18:22.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803131+0000", 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:22.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803138+0000", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9890000000000003e-06 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803144+0000", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9059999999999999e-06 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.803149+0000", 2024-09-16T22:18:22.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847055+0000", 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043905782999999997 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847129+0000", 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4203999999999996e-05 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-16T22:18:22.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-16T22:17:08.847194+0000", 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4448000000000004e-05 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-16T22:18:22.705 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-16T22:18:22.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T22:18:22.707 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:22.708+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:22.807 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:22.809+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:22.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T22:18:22.908 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-16T22:18:22.910+0000 7f44f4554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33621) UID: 0 2024-09-16T22:18:22.971 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:22.971 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:22.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T22:18:23.009 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:23.011+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:23.110 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:23.112+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:23.128 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:115257 3.f 3:f21cd0a5:::rbd_data.11cd289e90a9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.067541012999996, 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060899096, 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115257 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062060+0000", 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.062062+0000", 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.654e-06 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067529+0000", 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054671850000000003 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067549+0000", 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9890000000000001e-05 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067557+0000", 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067621+0000", 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3548999999999998e-05 2024-09-16T22:18:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.067787+0000", 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016645599999999999 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.068468+0000", 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000680458 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122600+0000", 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054132508000000003 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122861+0000", 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026092500000000001 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122908+0000", 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7231000000000002e-05 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:12.122959+0000", 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0519000000000002e-05 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4557.0:185386 3.e e84/23 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head v 84'381, mlcod=84'379)", 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.479316871000002, 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068943283999999994, 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650284+0000", 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650285+0000", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650302+0000", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6863000000000001e-05 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650311+0000", 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7030000000000001e-06 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650318+0000", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4880000000000004e-06 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650367+0000", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8816000000000002e-05 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650379+0000", 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2193e-05 2024-09-16T22:18:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719179+0000", 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068799948 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719227+0000", 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8575999999999997e-05 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209100 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.743100884, 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048430333999999998, 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209100 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386500+0000", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386501+0000", 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-16T22:18:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386503+0000", 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386509+0000", 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386514+0000", 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434770+0000", 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048256169000000002 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434853+0000", 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3564000000000002e-05 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434930+0000", 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6965000000000006e-05 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209101 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1300480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.743080460000002, 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048632158000000002, 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209101 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386520+0000", 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386521+0000", 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386523+0000", 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386529+0000", 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0279999999999997e-06 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386534+0000", 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.011e-06 2024-09-16T22:18:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.434975+0000", 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048440730000000001 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435066+0000", 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0781999999999999e-05 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435153+0000", 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7021999999999995e-05 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209105 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.743060452999998, 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049137526000000001, 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209105 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386540+0000", 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386541+0000", 2024-09-16T22:18:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386543+0000", 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9360000000000002e-06 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386549+0000", 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386554+0000", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435316+0000", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048762356 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435501+0000", 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018513700000000001 2024-09-16T22:18:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435678+0000", 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017650000000000001 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209107 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1398784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.743040168999997, 2024-09-16T22:18:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049257239000000001, 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209107 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386561+0000", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386563+0000", 2024-09-16T22:18:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8360000000000001e-06 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386569+0000", 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.268e-06 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386574+0000", 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5809999999999998e-06 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435712+0000", 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049138371 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435774+0000", 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1867999999999998e-05 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435818+0000", 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3909000000000003e-05 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209108 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1421312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742869931999998, 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049180447000000002, 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209108 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386731+0000", 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386732+0000", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386735+0000", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386744+0000", 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8480000000000007e-06 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386750+0000", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1310000000000003e-06 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435835+0000", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049085128999999998 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435873+0000", 2024-09-16T22:18:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8238999999999998e-05 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435911+0000", 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8068000000000002e-05 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209109 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742840039999997, 2024-09-16T22:18:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049312382000000002, 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209109 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386761+0000", 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386764+0000", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.61e-06 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386771+0000", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386776+0000", 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0769999999999998e-06 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.435945+0000", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049169088999999999 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436021+0000", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999994e-05 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436073+0000", 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2224e-05 2024-09-16T22:18:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209111 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1280000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742816402999999, 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049462918000000002, 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209111 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386784+0000", 2024-09-16T22:18:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386785+0000", 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386787+0000", 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2469999999999998e-06 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386794+0000", 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386799+0000", 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436094+0000", 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049294627000000001 2024-09-16T22:18:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436172+0000", 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8532e-05 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436247+0000", 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5030000000000005e-05 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209113 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1302528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742794338000003, 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049562408000000002, 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209113 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386807+0000", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386809+0000", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386815+0000", 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.404e-06 2024-09-16T22:18:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386820+0000", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9799999999999998e-06 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436274+0000", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049453338999999999 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436325+0000", 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1282999999999997e-05 2024-09-16T22:18:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436369+0000", 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3961000000000001e-05 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209115 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1378304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742773407999998, 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049705634999999998, 2024-09-16T22:18:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209115 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386827+0000", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386828+0000", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386835+0000", 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386842+0000", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9009999999999997e-06 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386847+0000", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8690000000000003e-06 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436398+0000", 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049551928000000002 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436474+0000", 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5426999999999995e-05 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436533+0000", 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9236e-05 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209116 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1400832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742746564000001, 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049851050000000001, 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209116 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386854+0000", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386855+0000", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386857+0000", 2024-09-16T22:18:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2350000000000002e-06 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386864+0000", 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386868+0000", 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3719999999999998e-06 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436554+0000", 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049685512000000001 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436631+0000", 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7101999999999995e-05 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436705+0000", 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4672000000000006e-05 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209117 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1259520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742725198000002, 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049982442000000002, 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209117 2024-09-16T22:18:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386876+0000", 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386878+0000", 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386884+0000", 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1469999999999998e-06 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386889+0000", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436728+0000", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049839120000000001 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436783+0000", 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4651000000000002e-05 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436858+0000", 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5018000000000006e-05 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209118 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742704402999998, 2024-09-16T22:18:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050142071000000003, 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209118 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386896+0000", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386897+0000", 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386899+0000", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386905+0000", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-06 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386910+0000", 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6290000000000001e-06 2024-09-16T22:18:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436890+0000", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049980199000000003 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.436960+0000", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0119999999999999e-05 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437039+0000", 2024-09-16T22:18:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8122999999999997e-05 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209119 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742677694999998, 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050313031000000001, 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209119 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386923+0000", 2024-09-16T22:18:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386924+0000", 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386927+0000", 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8329999999999998e-06 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386936+0000", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2289999999999992e-06 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386941+0000", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.237e-06 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437072+0000", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050131241 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437158+0000", 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5432000000000005e-05 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437236+0000", 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8526e-05 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209121 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1304576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742651504000001, 2024-09-16T22:18:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050469370999999999, 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209121 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386949+0000", 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386950+0000", 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386952+0000", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386958+0000", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2570000000000004e-06 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386963+0000", 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.775e-06 2024-09-16T22:18:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437268+0000", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050305018 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437351+0000", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3230000000000001e-05 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437419+0000", 2024-09-16T22:18:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7267999999999994e-05 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209123 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1380352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742629772000001, 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050625205999999999, 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209123 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386971+0000", 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386972+0000", 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386974+0000", 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9470000000000002e-06 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386980+0000", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386985+0000", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9280000000000001e-06 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437452+0000", 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050466715000000002 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437533+0000", 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0950000000000003e-05 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437596+0000", 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3752999999999998e-05 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209124 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742608840999999, 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050781967999999997, 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209124 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386992+0000", 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-16T22:18:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.386995+0000", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0490000000000002e-06 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387001+0000", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387005+0000", 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.279e-06 2024-09-16T22:18:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437625+0000", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050619999999999998 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437699+0000", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3654999999999999e-05 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437774+0000", 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5228000000000005e-05 2024-09-16T22:18:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209125 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742588304999998, 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050959480000000001, 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209125 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387013+0000", 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387023+0000", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0145e-05 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387026+0000", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2650000000000001e-06 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387032+0000", 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.071e-06 2024-09-16T22:18:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387037+0000", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0939999999999998e-06 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437806+0000", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050768501000000001 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437884+0000", 2024-09-16T22:18:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8670000000000004e-05 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.437972+0000", 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7733999999999997e-05 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:209126 3.c 3:3161d1f3:::rbd_data.11cd60092c51.000000000000000c:2e [read 1284096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.742555775, 2024-09-16T22:18:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051107410999999998, 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 209126 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.387045+0000", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393010+0000", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059651950000000004 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393016+0000", 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8819999999999998e-06 2024-09-16T22:18:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393025+0000", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7770000000000008e-06 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.393033+0000", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438007+0000", 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044974440999999997 2024-09-16T22:18:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438084+0000", 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7477000000000004e-05 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.438153+0000", 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8021999999999993e-05 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.222 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:23.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T22:18:23.224 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:18:23.224 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:23.224+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:23.323 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-16T22:18:23.325+0000 7f5f74156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33614) UID: 0 2024-09-16T22:18:23.374 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:23.375 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:23.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T22:18:23.424 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:23.426+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:23.526 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:23.527+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:23.530 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:23.531 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:23.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T22:18:23.627 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:23.629+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195591 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 137216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106787969999999, 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077332743999999995, 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195591 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582238+0000", 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-16T22:18:23.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582240+0000", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.714e-06 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582247+0000", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1940000000000001e-06 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582252+0000", 2024-09-16T22:18:23.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6619999999999996e-06 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659350+0000", 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077097984999999994 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659399+0000", 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9243000000000003e-05 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659570+0000", 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017159400000000001 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195592 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 159744~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106768228, 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077529482999999996, 2024-09-16T22:18:23.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195592 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582257+0000", 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582258+0000", 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582266+0000", 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0819999999999999e-06 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582273+0000", 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4569999999999999e-06 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582278+0000", 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2379999999999997e-06 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659590+0000", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077311991999999996 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659638+0000", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7348999999999997e-05 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659787+0000", 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148994 2024-09-16T22:18:23.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195593 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 182272~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106738778, 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077727187000000003, 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195593 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582287+0000", 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582289+0000", 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582297+0000", 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1310000000000006e-06 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582303+0000", 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5169999999999999e-06 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659805+0000", 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077501808000000005 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.659847+0000", 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2111999999999999e-05 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660014+0000", 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016712699999999999 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195594 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 204800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106713151000001, 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077998578999999998, 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195594 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582312+0000", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582313+0000", 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-16T22:18:23.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582315+0000", 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.446e-06 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582323+0000", 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9570000000000002e-06 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582328+0000", 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660051+0000", 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077723061999999996 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660121+0000", 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9261e-05 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660311+0000", 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019047199999999999 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195595 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 227328~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106690497999999, 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078204709999999997, 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195595 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582335+0000", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582338+0000", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3269999999999999e-06 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582346+0000", 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8280000000000003e-06 2024-09-16T22:18:23.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582351+0000", 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660326+0000", 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077975059999999999 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660368+0000", 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1825e-05 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660540+0000", 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017182499999999999 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195596 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 249856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:23.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106667520000002, 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.07845597, 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195596 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582358+0000", 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582361+0000", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582368+0000", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7230000000000004e-06 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582374+0000", 2024-09-16T22:18:23.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3059999999999997e-06 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660573+0000", 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078198939999999995 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660617+0000", 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660814+0000", 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019657100000000001 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195597 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 280576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106644467000002, 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078795981000000001, 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195597 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582381+0000", 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582382+0000", 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582384+0000", 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582392+0000", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582397+0000", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660849+0000", 2024-09-16T22:18:23.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078452339999999995 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.660903+0000", 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3588000000000003e-05 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661177+0000", 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000274417 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195598 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 303104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106622747000003, 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079186263000000007, 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195598 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582403+0000", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582405+0000", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582412+0000", 2024-09-16T22:18:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2819999999999999e-06 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582417+0000", 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-06 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661252+0000", 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078834641999999996 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661339+0000", 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7162000000000003e-05 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661589+0000", 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025001299999999999 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195599 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 325632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106602543000001, 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079510722000000006, 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195599 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582423+0000", 2024-09-16T22:18:23.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582425+0000", 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0310000000000001e-06 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582431+0000", 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9529999999999996e-06 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582436+0000", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8459999999999999e-06 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661637+0000", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079200800000000002 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661730+0000", 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2601000000000003e-05 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661934+0000", 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020411600000000001 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195600 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 348160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106583952000001, 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079790020000000003, 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195600 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582442+0000", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582444+0000", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8899999999999999e-06 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582451+0000", 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2069999999999998e-06 2024-09-16T22:18:23.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582456+0000", 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7099999999999998e-06 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.661971+0000", 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079515505 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662020+0000", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999000000000003e-05 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662232+0000", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021133799999999999 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195601 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 370688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:23.735 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.106562422000003, 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.080146220000000004, 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195601 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582463+0000", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582464+0000", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582466+0000", 2024-09-16T22:18:23.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582473+0000", 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4460000000000004e-06 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.582478+0000", 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662264+0000", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079785705999999998 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662355+0000", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1118000000000004e-05 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.662609+0000", 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025443500000000001 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195742 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101443326000002, 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077125427999999996, 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195742 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587582+0000", 2024-09-16T22:18:23.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587583+0000", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587585+0000", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0080000000000001e-06 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587591+0000", 2024-09-16T22:18:23.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3110000000000002e-06 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587596+0000", 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664538+0000", 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.076941915999999999 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664613+0000", 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5186999999999997e-05 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664708+0000", 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4629000000000005e-05 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195743 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 610304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.744 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101423056999998, 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077952304, 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195743 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587603+0000", 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587605+0000", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587612+0000", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587617+0000", 2024-09-16T22:18:23.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.664738+0000", 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077121198000000002 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665451+0000", 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071260200000000001 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665555+0000", 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104045 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195744 3.10 3:0f66a368:::rbd_data.11cd60092c51.0000000000000009:2e [read 632832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101400927, 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078122217999999993, 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195744 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587625+0000", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587627+0000", 2024-09-16T22:18:23.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8589999999999999e-06 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587633+0000", 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587639+0000", 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000002e-06 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665587+0000", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077948633000000003 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665668+0000", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0421999999999997e-05 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665747+0000", 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9301999999999994e-05 2024-09-16T22:18:23.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195745 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 403456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101381445999998, 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078329731, 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195745 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587644+0000", 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4900000000000001e-07 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587651+0000", 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587657+0000", 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3210000000000001e-06 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587662+0000", 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665776+0000", 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078113314000000003 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665881+0000", 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105536 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.665974+0000", 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2632000000000004e-05 2024-09-16T22:18:23.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195746 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101357106000002, 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078495308, 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195746 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587668+0000", 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587669+0000", 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587671+0000", 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9269999999999999e-06 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587677+0000", 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6640000000000003e-06 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587682+0000", 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6370000000000004e-06 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666004+0000", 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078321842000000003 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666081+0000", 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7088999999999993e-05 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666164+0000", 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2689999999999996e-05 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195747 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101337139999998, 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078674441999999997, 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195747 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587688+0000", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587689+0000", 2024-09-16T22:18:23.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587691+0000", 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9609999999999999e-06 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587697+0000", 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587702+0000", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666192+0000", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078489929999999999 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666268+0000", 2024-09-16T22:18:23.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5947999999999999e-05 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666363+0000", 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4919000000000004e-05 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195748 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101315729, 2024-09-16T22:18:23.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078839111000000003, 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195748 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587710+0000", 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587713+0000", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587720+0000", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8029999999999999e-06 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587725+0000", 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.378e-06 2024-09-16T22:18:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666392+0000", 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078667313000000003 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666467+0000", 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4160999999999997e-05 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666549+0000", 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2397000000000004e-05 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195749 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101293968, 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078998926999999997, 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195749 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587732+0000", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-07 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587734+0000", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9290000000000001e-06 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587740+0000", 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2589999999999997e-06 2024-09-16T22:18:23.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587745+0000", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666577+0000", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078831632999999998 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666652+0000", 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-05 2024-09-16T22:18:23.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666731+0000", 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8296999999999999e-05 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:195750 3.10 3:0f282be3:::rbd_data.11cd60092c51.000000000000000a:2e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.101273499999998, 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.079158416999999995, 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:23.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 195750 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:23.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587752+0000", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587753+0000", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587755+0000", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:23.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587762+0000", 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4649999999999999e-06 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.587767+0000", 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666759+0000", 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078991546999999995 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666832+0000", 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3084999999999996e-05 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:35.666910+0000", 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8831999999999993e-05 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:23.781 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:23.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T22:18:23.783 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:23.785+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:23.884 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:23.885+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:23.891 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:23.891 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:23.891 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:23.891 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:23.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T22:18:23.985 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:23.987+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:24.047 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-16T22:18:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-16T22:18:24.048 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:24.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T22:18:24.085 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-16T22:18:24.088+0000 7f49e6060640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33330) UID: 0 2024-09-16T22:18:24.168 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:24.168 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-16T22:18:24.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45024 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.592654655999993, 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051725314000000001, 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45024 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577010+0000", 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577011+0000", 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577013+0000", 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577018+0000", 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-09-16T22:18:24.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577023+0000", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628570+0000", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051546514000000002 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628650+0000", 2024-09-16T22:18:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0448e-05 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628735+0000", 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5389000000000005e-05 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45025 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:24.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.592635315999999, 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051926617000000001, 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45025 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:24.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577029+0000", 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577404+0000", 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037412300000000002 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577407+0000", 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8139999999999998e-06 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577417+0000", 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1220000000000001e-06 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577423+0000", 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6610000000000003e-06 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628780+0000", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051356671999999999 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628863+0000", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3282999999999995e-05 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628956+0000", 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2942000000000006e-05 2024-09-16T22:18:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45026 3.e 3:767f8114:::rbd_data.11cd43ed039f.000000000000000d:16 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.592230916000005, 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051720724000000003, 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45026 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577434+0000", 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.46e-07 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577441+0000", 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3890000000000002e-06 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577447+0000", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5400000000000001e-06 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.577453+0000", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.628997+0000", 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051544341 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629072+0000", 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4575999999999999e-05 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.629155+0000", 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2978000000000004e-05 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45049 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.590535646000006, 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051857155000000002, 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45049 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579129+0000", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579130+0000", 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-16T22:18:24.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579132+0000", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579147+0000", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4878999999999999e-05 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579152+0000", 2024-09-16T22:18:24.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630823+0000", 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051671304000000001 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630889+0000", 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5392e-05 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.630986+0000", 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7455999999999997e-05 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45050 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.590505113000006, 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05204607, 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45050 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.579160+0000", 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582939+0000", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037795089999999999 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582945+0000", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582957+0000", 2024-09-16T22:18:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2092999999999999e-05 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.582964+0000", 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2679999999999997e-06 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631030+0000", 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048065373000000002 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631111+0000", 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1176999999999999e-05 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.631206+0000", 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4909999999999998e-05 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45069 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.586104214000002, 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051679322, 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45069 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583561+0000", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583563+0000", 2024-09-16T22:18:24.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583570+0000", 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4819999999999999e-06 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583574+0000", 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-06 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635077+0000", 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051502522000000002 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635149+0000", 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729000000000004e-05 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635240+0000", 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1329000000000006e-05 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45070 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 102400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.586083732999995, 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051876685999999998, 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45070 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583581+0000", 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583583+0000", 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583590+0000", 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583595+0000", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1170000000000003e-06 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635282+0000", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051686966000000001 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635360+0000", 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999994e-05 2024-09-16T22:18:24.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635458+0000", 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7573999999999998e-05 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45071 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.586063156999998, 2024-09-16T22:18:24.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052054617999999997, 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45071 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583602+0000", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583604+0000", 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.928e-06 2024-09-16T22:18:24.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583610+0000", 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4779999999999996e-06 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583619+0000", 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635499+0000", 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051879941999999998 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635572+0000", 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2867999999999994e-05 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635656+0000", 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45072 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 118784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.586038672000001, 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052248050999999997, 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45072 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583626+0000", 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583627+0000", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583629+0000", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-06 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583635+0000", 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7580000000000003e-06 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583641+0000", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635697+0000", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052055754000000003 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635780+0000", 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3244999999999993e-05 2024-09-16T22:18:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635874+0000", 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4300000000000002e-05 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45073 3.16 3:6e73e92f:::rbd_data.11cd43ed039f.0000000000000017:16 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.586017100999996, 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052439604000000001, 2024-09-16T22:18:24.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45073 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.583648+0000", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588942+0000", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052940979999999997 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588948+0000", 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4130000000000003e-06 2024-09-16T22:18:24.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588960+0000", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1538e-05 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.588969+0000", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1940000000000004e-06 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.635916+0000", 2024-09-16T22:18:24.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046947164 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636002+0000", 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5484999999999999e-05 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.636087+0000", 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5711999999999995e-05 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45148 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579938502999994, 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051867178999999999, 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45148 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589726+0000", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589727+0000", 2024-09-16T22:18:24.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589729+0000", 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0279999999999999e-06 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589735+0000", 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2890000000000003e-06 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589740+0000", 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641421+0000", 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051681284000000001 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641495+0000", 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000006e-05 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641593+0000", 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8066999999999995e-05 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45149 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579918171000003, 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052053704999999999, 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45149 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589747+0000", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589749+0000", 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9489999999999999e-06 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589756+0000", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589761+0000", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5459999999999999e-06 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641634+0000", 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051872424 2024-09-16T22:18:24.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641708+0000", 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3919999999999997e-05 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641800+0000", 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2749000000000003e-05 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45150 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579896951999999, 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052252521000000003, 2024-09-16T22:18:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45150 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589768+0000", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589770+0000", 2024-09-16T22:18:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589776+0000", 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589782+0000", 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4519999999999996e-06 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641843+0000", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052061481999999999 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.641926+0000", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2619000000000003e-05 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642020+0000", 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4767999999999998e-05 2024-09-16T22:18:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45151 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579875580000007, 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052432759000000002, 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45151 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589789+0000", 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589790+0000", 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589792+0000", 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589798+0000", 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589803+0000", 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642059+0000", 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052255089999999997 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642133+0000", 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4666000000000006e-05 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642222+0000", 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8774000000000001e-05 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45152 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 88064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579854741999995, 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052651417999999998, 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45152 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589810+0000", 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589813+0000", 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589819+0000", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589824+0000", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.023e-06 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642262+0000", 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052438179000000001 2024-09-16T22:18:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642342+0000", 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9414999999999998e-05 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642461+0000", 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011993399999999999 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45153 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 96256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579834160999994, 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052783669999999998, 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45153 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589831+0000", 2024-09-16T22:18:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589833+0000", 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589840+0000", 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589845+0000", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1569999999999999e-06 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642514+0000", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052669042999999999 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642558+0000", 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3773999999999998e-05 2024-09-16T22:18:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642614+0000", 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6456000000000003e-05 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45154 3.16 3:6950f6e5:::rbd_data.11cd43ed039f.0000000000000016:16 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.579812594000003, 2024-09-16T22:18:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052965482000000001, 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45154 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.589852+0000", 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593519+0000", 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036672200000000001 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593529+0000", 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0005e-05 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593543+0000", 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3859999999999999e-05 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.593561+0000", 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642652+0000", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049091057 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642726+0000", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3707000000000004e-05 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.642818+0000", 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2091999999999999e-05 2024-09-16T22:18:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:45548 3.e 3:7563b8b8:::rbd_data.11cd43ed039f.000000000000011c:16 [read 20480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.575165720000001, 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051788016999999999, 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 45548 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.594499+0000", 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596267+0000", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001767926 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596272+0000", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3489999999999999e-06 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596285+0000", 2024-09-16T22:18:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-05 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.596292+0000", 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646115+0000", 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049822389000000002 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646186+0000", 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1298999999999996e-05 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:16:50.646287+0000", 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101037 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:185386 3.e 3:778c60e7:::rbd_data.11cded7d7aa9.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write 21630569~10940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T22:18:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.519860319000003, 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069680716000000004, 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 185386 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649804+0000", 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649806+0000", 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649818+0000", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2064e-05 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649836+0000", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7385e-05 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649843+0000", 2024-09-16T22:18:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.481e-06 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.649914+0000", 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0821999999999994e-05 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650033+0000", 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118874 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-16T22:18:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.650188+0000", 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155277 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.651108+0000", 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092009899999999996 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719408+0000", 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068299905999999994 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719451+0000", 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2616e-05 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:31.719485+0000", 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4461000000000001e-05 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4557.0:207744 3.16 3:6e867e10:::rbd_data.11cdd6518f4.0000000000000017:2f [read 215040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.842671336000002, 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051713172000000002, 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4557", 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1962029417", 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 207744 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-16T22:18:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326993+0000", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326994+0000", 2024-09-16T22:18:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.326996+0000", 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328803+0000", 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018065710000000001 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.328814+0000", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1618e-05 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378353+0000", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049538304999999998 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378427+0000", 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4169000000000001e-05 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T22:18:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-16T22:17:36.378707+0000", 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027975500000000002 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T22:18:24.616 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:24.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T22:18:24.618 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:24.619+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:24.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T22:18:24.718 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-16T22:18:24.720+0000 7f7e6b5b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33329) UID: 0 2024-09-16T22:18:24.770 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:24.770 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-16T22:18:24.770 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-16T22:18:24.770 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:24.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T22:18:24.786 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-16T22:18:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T22:18:24.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-16T22:18:24.804 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T22:18:24.819 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:24.821+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:24.820 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-16T22:18:24.821 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T22:18:24.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-16T22:18:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T22:18:24.855 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-16T22:18:24.855 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T22:18:24.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-16T22:18:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T22:18:24.904 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:24.904 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:24.906 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-16T22:18:24.906 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T22:18:24.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T22:18:24.920 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:24.922+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:24.926 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-16T22:18:24.927 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T22:18:25.021 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-16T22:18:25.022+0000 7f479bec1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33623) UID: 0 2024-09-16T22:18:25.033 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.033 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T22:18:25.122 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-16T22:18:25.123+0000 7f46bf156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33326) UID: 0 2024-09-16T22:18:25.153 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.153 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T22:18:25.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T22:18:25.245 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-16T22:18:25.247+0000 7f29e96cf640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33322) UID: 0 2024-09-16T22:18:25.274 INFO:teuthology.orchestra.run.smithi031.stdout:All operations completed A-OK! 2024-09-16T22:18:25.290 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:25.290 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T22:18:25.300 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T22:18:25.326 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T22:18:25.327 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.327 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T22:18:25.471 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.472 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T22:18:25.583 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.583 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T22:18:25.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T22:18:25.693 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.693 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T22:18:25.794 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-16T22:18:25.795 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:18:25.804 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-16T22:18:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:18:25.808 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.808 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.812 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-16T22:18:25.813 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T22:18:25.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T22:18:25.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T22:18:25.919 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:25.919 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:25.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T22:18:26.030 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:26.031 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:26.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T22:18:26.141 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:26.141 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:26.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T22:18:26.187 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-16T22:18:26.190 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T22:18:26.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T22:18:26.254 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:26.254 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:26.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T22:18:26.366 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:26.366 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:26.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T22:18:27.196 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T22:18:27.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T22:18:28.201 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T22:18:28.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T22:18:28.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T22:18:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T22:18:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-16T22:18:29.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T22:18:29.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T22:18:30.209 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-16T22:18:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T22:18:30.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T22:18:31.219 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T22:18:31.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T22:18:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T22:18:32.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T22:18:33.237 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T22:18:33.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T22:18:34.137 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T22:18:34.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T22:18:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T22:18:35.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T22:18:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T22:18:36.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T22:18:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T22:18:37.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T22:18:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T22:18:38.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T22:18:39.273 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T22:18:39.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T22:18:40.277 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T22:18:40.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T22:18:41.283 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T22:18:41.301 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T22:18:41.302 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T22:18:41.302 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T22:18:41.302 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T22:18:41.302 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T22:18:41.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:18:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":153,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:18:41.283657+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T22:16:02.234097+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T22:16:13.789572+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:18:41.621 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T22:18:41.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:18:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:41.922 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":153,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:18:41.283657+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T22:16:02.234097+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T22:16:13.789572+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:18:41.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T22:18:41.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T22:18:41.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T22:18:41.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T22:18:41.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T22:18:41.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T22:18:41.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T22:18:41.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T22:18:42.188 INFO:teuthology.orchestra.run.smithi031.stdout:64424509477 2024-09-16T22:18:42.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T22:18:42.193 INFO:teuthology.orchestra.run.smithi031.stdout:64424509477 2024-09-16T22:18:42.194 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:42.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T22:18:42.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T22:18:42.195 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:42.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T22:18:42.197 INFO:teuthology.orchestra.run.smithi031.stdout:55834574885 2024-09-16T22:18:42.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T22:18:42.198 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:42.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T22:18:42.199 INFO:teuthology.orchestra.run.smithi031.stdout:64424509479 2024-09-16T22:18:42.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T22:18:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:42.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T22:18:42.777 INFO:teuthology.orchestra.run.smithi031.stdout:64424509475 2024-09-16T22:18:42.796 INFO:teuthology.orchestra.run.smithi031.stdout:64424509476 2024-09-16T22:18:42.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509477 got 64424509475 for osd.3 2024-09-16T22:18:42.812 INFO:teuthology.orchestra.run.smithi031.stdout:64424509476 2024-09-16T22:18:42.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509476 for osd.2 2024-09-16T22:18:42.821 INFO:teuthology.orchestra.run.smithi031.stdout:64424509475 2024-09-16T22:18:42.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509476 for osd.6 2024-09-16T22:18:42.836 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-09-16T22:18:42.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509477 got 64424509475 for osd.0 2024-09-16T22:18:42.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574885 got 55834574884 for osd.7 2024-09-16T22:18:42.854 INFO:teuthology.orchestra.run.smithi031.stdout:64424509476 2024-09-16T22:18:42.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509476 for osd.4 2024-09-16T22:18:42.869 INFO:teuthology.orchestra.run.smithi031.stdout:64424509476 2024-09-16T22:18:42.874 INFO:teuthology.orchestra.run.smithi031.stdout:64424509477 2024-09-16T22:18:42.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509476 for osd.5 2024-09-16T22:18:42.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509477 for osd.1 2024-09-16T22:18:43.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T22:18:43.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T22:18:43.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T22:18:43.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T22:18:43.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T22:18:43.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T22:18:43.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T22:18:43.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T22:18:44.393 INFO:teuthology.orchestra.run.smithi031.stdout:64424509477 2024-09-16T22:18:44.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509477 got 64424509477 for osd.3 2024-09-16T22:18:44.411 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.444 INFO:teuthology.orchestra.run.smithi031.stdout:64424509477 2024-09-16T22:18:44.455 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:44.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509477 got 64424509477 for osd.0 2024-09-16T22:18:44.463 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.467 INFO:teuthology.orchestra.run.smithi031.stdout:55834574885 2024-09-16T22:18:44.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509478 for osd.2 2024-09-16T22:18:44.475 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509478 for osd.6 2024-09-16T22:18:44.479 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574885 got 55834574885 for osd.7 2024-09-16T22:18:44.484 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.491 INFO:teuthology.orchestra.run.smithi031.stdout:64424509479 2024-09-16T22:18:44.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.1 2024-09-16T22:18:44.505 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:44.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509478 for osd.5 2024-09-16T22:18:44.544 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.545 INFO:teuthology.orchestra.run.smithi031.stdout:64424509478 2024-09-16T22:18:44.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509478 got 64424509478 for osd.4 2024-09-16T22:18:44.559 DEBUG:teuthology.parallel:result is None 2024-09-16T22:18:44.559 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T22:18:44.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:44.899 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:44.910 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T22:18:44.598336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371852,"kb_used_data":15420,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602868,"statfs":{"total":773094113280,"available":772713336832,"internally_reserved":0,"allocated":15790080,"data_stored":8115499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001291"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3054,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288827+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288827+0000","last_peered":"2024-09-16T22:18:41.288827+0000","last_clean":"2024-09-16T22:18:41.288827+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288827+0000","last_undegraded":"2024-09-16T22:18:41.288827+0000","last_fullsized":"2024-09-16T22:18:41.288827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6350,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249768+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:41.249768+0000","last_peered":"2024-09-16T22:18:41.249768+0000","last_clean":"2024-09-16T22:18:41.249768+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:41.249768+0000","last_undegraded":"2024-09-16T22:18:41.249768+0000","last_fullsized":"2024-09-16T22:18:41.249768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7246,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165166+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:41.165166+0000","last_peered":"2024-09-16T22:18:41.165166+0000","last_clean":"2024-09-16T22:18:41.165166+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:41.165166+0000","last_undegraded":"2024-09-16T22:18:41.165166+0000","last_fullsized":"2024-09-16T22:18:41.165166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22402,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249985+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:41.249985+0000","last_peered":"2024-09-16T22:18:41.249985+0000","last_clean":"2024-09-16T22:18:41.249985+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:41.249985+0000","last_undegraded":"2024-09-16T22:18:41.249985+0000","last_fullsized":"2024-09-16T22:18:41.249985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165208+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:41.165208+0000","last_peered":"2024-09-16T22:18:41.165208+0000","last_clean":"2024-09-16T22:18:41.165208+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:41.165208+0000","last_undegraded":"2024-09-16T22:18:41.165208+0000","last_fullsized":"2024-09-16T22:18:41.165208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249841+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:41.249841+0000","last_peered":"2024-09-16T22:18:41.249841+0000","last_clean":"2024-09-16T22:18:41.249841+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:41.249841+0000","last_undegraded":"2024-09-16T22:18:41.249841+0000","last_fullsized":"2024-09-16T22:18:41.249841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2794,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282330+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:40.282330+0000","last_peered":"2024-09-16T22:18:40.282330+0000","last_clean":"2024-09-16T22:18:40.282330+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:40.282330+0000","last_undegraded":"2024-09-16T22:18:40.282330+0000","last_fullsized":"2024-09-16T22:18:40.282330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13679,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165219+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:41.165219+0000","last_peered":"2024-09-16T22:18:41.165219+0000","last_clean":"2024-09-16T22:18:41.165219+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:41.165219+0000","last_undegraded":"2024-09-16T22:18:41.165219+0000","last_fullsized":"2024-09-16T22:18:41.165219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165245+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:41.165245+0000","last_peered":"2024-09-16T22:18:41.165245+0000","last_clean":"2024-09-16T22:18:41.165245+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:41.165245+0000","last_undegraded":"2024-09-16T22:18:41.165245+0000","last_fullsized":"2024-09-16T22:18:41.165245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165239+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:41.165239+0000","last_peered":"2024-09-16T22:18:41.165239+0000","last_clean":"2024-09-16T22:18:41.165239+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:41.165239+0000","last_undegraded":"2024-09-16T22:18:41.165239+0000","last_fullsized":"2024-09-16T22:18:41.165239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2669,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8903,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281931+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:40.281931+0000","last_peered":"2024-09-16T22:18:40.281931+0000","last_clean":"2024-09-16T22:18:40.281931+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:40.281931+0000","last_undegraded":"2024-09-16T22:18:40.281931+0000","last_fullsized":"2024-09-16T22:18:40.281931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2831,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165131+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:41.165131+0000","last_peered":"2024-09-16T22:18:41.165131+0000","last_clean":"2024-09-16T22:18:41.165131+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:41.165131+0000","last_undegraded":"2024-09-16T22:18:41.165131+0000","last_fullsized":"2024-09-16T22:18:41.165131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":328,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249804+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:41.249804+0000","last_peered":"2024-09-16T22:18:41.249804+0000","last_clean":"2024-09-16T22:18:41.249804+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:41.249804+0000","last_undegraded":"2024-09-16T22:18:41.249804+0000","last_fullsized":"2024-09-16T22:18:41.249804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4369,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165125+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:41.165125+0000","last_peered":"2024-09-16T22:18:41.165125+0000","last_clean":"2024-09-16T22:18:41.165125+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:41.165125+0000","last_undegraded":"2024-09-16T22:18:41.165125+0000","last_fullsized":"2024-09-16T22:18:41.165125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3385,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282139+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:40.282139+0000","last_peered":"2024-09-16T22:18:40.282139+0000","last_clean":"2024-09-16T22:18:40.282139+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:40.282139+0000","last_undegraded":"2024-09-16T22:18:40.282139+0000","last_fullsized":"2024-09-16T22:18:40.282139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6647,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249819+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:41.249819+0000","last_peered":"2024-09-16T22:18:41.249819+0000","last_clean":"2024-09-16T22:18:41.249819+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:41.249819+0000","last_undegraded":"2024-09-16T22:18:41.249819+0000","last_fullsized":"2024-09-16T22:18:41.249819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5785,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281952+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:40.281952+0000","last_peered":"2024-09-16T22:18:40.281952+0000","last_clean":"2024-09-16T22:18:40.281952+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:40.281952+0000","last_undegraded":"2024-09-16T22:18:40.281952+0000","last_fullsized":"2024-09-16T22:18:40.281952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12343,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51600,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320240,"statfs":{"total":96636764160,"available":96583925760,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:44.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:45.223 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T22:18:44.598336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371852,"kb_used_data":15420,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602868,"statfs":{"total":773094113280,"available":772713336832,"internally_reserved":0,"allocated":15790080,"data_stored":8115499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001291"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3054,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288827+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288827+0000","last_peered":"2024-09-16T22:18:41.288827+0000","last_clean":"2024-09-16T22:18:41.288827+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288827+0000","last_undegraded":"2024-09-16T22:18:41.288827+0000","last_fullsized":"2024-09-16T22:18:41.288827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6350,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249768+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:41.249768+0000","last_peered":"2024-09-16T22:18:41.249768+0000","last_clean":"2024-09-16T22:18:41.249768+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:41.249768+0000","last_undegraded":"2024-09-16T22:18:41.249768+0000","last_fullsized":"2024-09-16T22:18:41.249768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7246,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165166+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:41.165166+0000","last_peered":"2024-09-16T22:18:41.165166+0000","last_clean":"2024-09-16T22:18:41.165166+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:41.165166+0000","last_undegraded":"2024-09-16T22:18:41.165166+0000","last_fullsized":"2024-09-16T22:18:41.165166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22402,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249985+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:41.249985+0000","last_peered":"2024-09-16T22:18:41.249985+0000","last_clean":"2024-09-16T22:18:41.249985+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:41.249985+0000","last_undegraded":"2024-09-16T22:18:41.249985+0000","last_fullsized":"2024-09-16T22:18:41.249985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165208+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:41.165208+0000","last_peered":"2024-09-16T22:18:41.165208+0000","last_clean":"2024-09-16T22:18:41.165208+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:41.165208+0000","last_undegraded":"2024-09-16T22:18:41.165208+0000","last_fullsized":"2024-09-16T22:18:41.165208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249841+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:41.249841+0000","last_peered":"2024-09-16T22:18:41.249841+0000","last_clean":"2024-09-16T22:18:41.249841+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:41.249841+0000","last_undegraded":"2024-09-16T22:18:41.249841+0000","last_fullsized":"2024-09-16T22:18:41.249841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2794,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282330+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:40.282330+0000","last_peered":"2024-09-16T22:18:40.282330+0000","last_clean":"2024-09-16T22:18:40.282330+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:40.282330+0000","last_undegraded":"2024-09-16T22:18:40.282330+0000","last_fullsized":"2024-09-16T22:18:40.282330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13679,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165219+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:41.165219+0000","last_peered":"2024-09-16T22:18:41.165219+0000","last_clean":"2024-09-16T22:18:41.165219+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:41.165219+0000","last_undegraded":"2024-09-16T22:18:41.165219+0000","last_fullsized":"2024-09-16T22:18:41.165219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165245+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:41.165245+0000","last_peered":"2024-09-16T22:18:41.165245+0000","last_clean":"2024-09-16T22:18:41.165245+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:41.165245+0000","last_undegraded":"2024-09-16T22:18:41.165245+0000","last_fullsized":"2024-09-16T22:18:41.165245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165239+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:41.165239+0000","last_peered":"2024-09-16T22:18:41.165239+0000","last_clean":"2024-09-16T22:18:41.165239+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:41.165239+0000","last_undegraded":"2024-09-16T22:18:41.165239+0000","last_fullsized":"2024-09-16T22:18:41.165239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2669,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8903,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281931+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:40.281931+0000","last_peered":"2024-09-16T22:18:40.281931+0000","last_clean":"2024-09-16T22:18:40.281931+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:40.281931+0000","last_undegraded":"2024-09-16T22:18:40.281931+0000","last_fullsized":"2024-09-16T22:18:40.281931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2831,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165131+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:41.165131+0000","last_peered":"2024-09-16T22:18:41.165131+0000","last_clean":"2024-09-16T22:18:41.165131+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:41.165131+0000","last_undegraded":"2024-09-16T22:18:41.165131+0000","last_fullsized":"2024-09-16T22:18:41.165131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":328,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249804+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:41.249804+0000","last_peered":"2024-09-16T22:18:41.249804+0000","last_clean":"2024-09-16T22:18:41.249804+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:41.249804+0000","last_undegraded":"2024-09-16T22:18:41.249804+0000","last_fullsized":"2024-09-16T22:18:41.249804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4369,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165125+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:41.165125+0000","last_peered":"2024-09-16T22:18:41.165125+0000","last_clean":"2024-09-16T22:18:41.165125+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:41.165125+0000","last_undegraded":"2024-09-16T22:18:41.165125+0000","last_fullsized":"2024-09-16T22:18:41.165125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3385,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282139+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:40.282139+0000","last_peered":"2024-09-16T22:18:40.282139+0000","last_clean":"2024-09-16T22:18:40.282139+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:40.282139+0000","last_undegraded":"2024-09-16T22:18:40.282139+0000","last_fullsized":"2024-09-16T22:18:40.282139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6647,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249819+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:41.249819+0000","last_peered":"2024-09-16T22:18:41.249819+0000","last_clean":"2024-09-16T22:18:41.249819+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:41.249819+0000","last_undegraded":"2024-09-16T22:18:41.249819+0000","last_fullsized":"2024-09-16T22:18:41.249819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5785,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281952+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:40.281952+0000","last_peered":"2024-09-16T22:18:40.281952+0000","last_clean":"2024-09-16T22:18:40.281952+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:40.281952+0000","last_undegraded":"2024-09-16T22:18:40.281952+0000","last_fullsized":"2024-09-16T22:18:40.281952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12343,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51600,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320240,"statfs":{"total":96636764160,"available":96583925760,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:45.225 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T22:18:45.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T22:18:45.545 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T22:18:45.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T22:18:45.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:45.854 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:45.855 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:45.867 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T22:18:44.598336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371852,"kb_used_data":15420,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602868,"statfs":{"total":773094113280,"available":772713336832,"internally_reserved":0,"allocated":15790080,"data_stored":8115499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001291"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3054,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288827+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288827+0000","last_peered":"2024-09-16T22:18:41.288827+0000","last_clean":"2024-09-16T22:18:41.288827+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288827+0000","last_undegraded":"2024-09-16T22:18:41.288827+0000","last_fullsized":"2024-09-16T22:18:41.288827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6350,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249768+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:41.249768+0000","last_peered":"2024-09-16T22:18:41.249768+0000","last_clean":"2024-09-16T22:18:41.249768+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:41.249768+0000","last_undegraded":"2024-09-16T22:18:41.249768+0000","last_fullsized":"2024-09-16T22:18:41.249768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7246,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165166+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:41.165166+0000","last_peered":"2024-09-16T22:18:41.165166+0000","last_clean":"2024-09-16T22:18:41.165166+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:41.165166+0000","last_undegraded":"2024-09-16T22:18:41.165166+0000","last_fullsized":"2024-09-16T22:18:41.165166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22402,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249985+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:41.249985+0000","last_peered":"2024-09-16T22:18:41.249985+0000","last_clean":"2024-09-16T22:18:41.249985+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:41.249985+0000","last_undegraded":"2024-09-16T22:18:41.249985+0000","last_fullsized":"2024-09-16T22:18:41.249985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165208+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:41.165208+0000","last_peered":"2024-09-16T22:18:41.165208+0000","last_clean":"2024-09-16T22:18:41.165208+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:41.165208+0000","last_undegraded":"2024-09-16T22:18:41.165208+0000","last_fullsized":"2024-09-16T22:18:41.165208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249841+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:41.249841+0000","last_peered":"2024-09-16T22:18:41.249841+0000","last_clean":"2024-09-16T22:18:41.249841+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:41.249841+0000","last_undegraded":"2024-09-16T22:18:41.249841+0000","last_fullsized":"2024-09-16T22:18:41.249841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2794,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282330+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:40.282330+0000","last_peered":"2024-09-16T22:18:40.282330+0000","last_clean":"2024-09-16T22:18:40.282330+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:40.282330+0000","last_undegraded":"2024-09-16T22:18:40.282330+0000","last_fullsized":"2024-09-16T22:18:40.282330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13679,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165219+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:41.165219+0000","last_peered":"2024-09-16T22:18:41.165219+0000","last_clean":"2024-09-16T22:18:41.165219+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:41.165219+0000","last_undegraded":"2024-09-16T22:18:41.165219+0000","last_fullsized":"2024-09-16T22:18:41.165219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165245+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:41.165245+0000","last_peered":"2024-09-16T22:18:41.165245+0000","last_clean":"2024-09-16T22:18:41.165245+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:41.165245+0000","last_undegraded":"2024-09-16T22:18:41.165245+0000","last_fullsized":"2024-09-16T22:18:41.165245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165239+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:41.165239+0000","last_peered":"2024-09-16T22:18:41.165239+0000","last_clean":"2024-09-16T22:18:41.165239+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:41.165239+0000","last_undegraded":"2024-09-16T22:18:41.165239+0000","last_fullsized":"2024-09-16T22:18:41.165239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2669,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8903,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281931+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:40.281931+0000","last_peered":"2024-09-16T22:18:40.281931+0000","last_clean":"2024-09-16T22:18:40.281931+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:40.281931+0000","last_undegraded":"2024-09-16T22:18:40.281931+0000","last_fullsized":"2024-09-16T22:18:40.281931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2831,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165131+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:41.165131+0000","last_peered":"2024-09-16T22:18:41.165131+0000","last_clean":"2024-09-16T22:18:41.165131+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:41.165131+0000","last_undegraded":"2024-09-16T22:18:41.165131+0000","last_fullsized":"2024-09-16T22:18:41.165131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":328,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249804+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:41.249804+0000","last_peered":"2024-09-16T22:18:41.249804+0000","last_clean":"2024-09-16T22:18:41.249804+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:41.249804+0000","last_undegraded":"2024-09-16T22:18:41.249804+0000","last_fullsized":"2024-09-16T22:18:41.249804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4369,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165125+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:41.165125+0000","last_peered":"2024-09-16T22:18:41.165125+0000","last_clean":"2024-09-16T22:18:41.165125+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:41.165125+0000","last_undegraded":"2024-09-16T22:18:41.165125+0000","last_fullsized":"2024-09-16T22:18:41.165125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3385,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282139+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:40.282139+0000","last_peered":"2024-09-16T22:18:40.282139+0000","last_clean":"2024-09-16T22:18:40.282139+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:40.282139+0000","last_undegraded":"2024-09-16T22:18:40.282139+0000","last_fullsized":"2024-09-16T22:18:40.282139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6647,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249819+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:41.249819+0000","last_peered":"2024-09-16T22:18:41.249819+0000","last_clean":"2024-09-16T22:18:41.249819+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:41.249819+0000","last_undegraded":"2024-09-16T22:18:41.249819+0000","last_fullsized":"2024-09-16T22:18:41.249819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5785,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281952+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:40.281952+0000","last_peered":"2024-09-16T22:18:40.281952+0000","last_clean":"2024-09-16T22:18:40.281952+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:40.281952+0000","last_undegraded":"2024-09-16T22:18:40.281952+0000","last_fullsized":"2024-09-16T22:18:40.281952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12343,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51600,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320240,"statfs":{"total":96636764160,"available":96583925760,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:45.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:46.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:46.177 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T22:18:44.598336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371852,"kb_used_data":15420,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602868,"statfs":{"total":773094113280,"available":772713336832,"internally_reserved":0,"allocated":15790080,"data_stored":8115499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001291"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3054,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288827+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288827+0000","last_peered":"2024-09-16T22:18:41.288827+0000","last_clean":"2024-09-16T22:18:41.288827+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288827+0000","last_undegraded":"2024-09-16T22:18:41.288827+0000","last_fullsized":"2024-09-16T22:18:41.288827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6350,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249768+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:41.249768+0000","last_peered":"2024-09-16T22:18:41.249768+0000","last_clean":"2024-09-16T22:18:41.249768+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:41.249768+0000","last_undegraded":"2024-09-16T22:18:41.249768+0000","last_fullsized":"2024-09-16T22:18:41.249768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7246,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165166+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:41.165166+0000","last_peered":"2024-09-16T22:18:41.165166+0000","last_clean":"2024-09-16T22:18:41.165166+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:41.165166+0000","last_undegraded":"2024-09-16T22:18:41.165166+0000","last_fullsized":"2024-09-16T22:18:41.165166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22402,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249985+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:41.249985+0000","last_peered":"2024-09-16T22:18:41.249985+0000","last_clean":"2024-09-16T22:18:41.249985+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:41.249985+0000","last_undegraded":"2024-09-16T22:18:41.249985+0000","last_fullsized":"2024-09-16T22:18:41.249985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165208+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:41.165208+0000","last_peered":"2024-09-16T22:18:41.165208+0000","last_clean":"2024-09-16T22:18:41.165208+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:41.165208+0000","last_undegraded":"2024-09-16T22:18:41.165208+0000","last_fullsized":"2024-09-16T22:18:41.165208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249841+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:41.249841+0000","last_peered":"2024-09-16T22:18:41.249841+0000","last_clean":"2024-09-16T22:18:41.249841+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:41.249841+0000","last_undegraded":"2024-09-16T22:18:41.249841+0000","last_fullsized":"2024-09-16T22:18:41.249841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2794,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282330+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:40.282330+0000","last_peered":"2024-09-16T22:18:40.282330+0000","last_clean":"2024-09-16T22:18:40.282330+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:40.282330+0000","last_undegraded":"2024-09-16T22:18:40.282330+0000","last_fullsized":"2024-09-16T22:18:40.282330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13679,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165219+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:41.165219+0000","last_peered":"2024-09-16T22:18:41.165219+0000","last_clean":"2024-09-16T22:18:41.165219+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:41.165219+0000","last_undegraded":"2024-09-16T22:18:41.165219+0000","last_fullsized":"2024-09-16T22:18:41.165219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165245+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:41.165245+0000","last_peered":"2024-09-16T22:18:41.165245+0000","last_clean":"2024-09-16T22:18:41.165245+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:41.165245+0000","last_undegraded":"2024-09-16T22:18:41.165245+0000","last_fullsized":"2024-09-16T22:18:41.165245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165239+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:41.165239+0000","last_peered":"2024-09-16T22:18:41.165239+0000","last_clean":"2024-09-16T22:18:41.165239+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:41.165239+0000","last_undegraded":"2024-09-16T22:18:41.165239+0000","last_fullsized":"2024-09-16T22:18:41.165239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2669,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8903,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281931+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:40.281931+0000","last_peered":"2024-09-16T22:18:40.281931+0000","last_clean":"2024-09-16T22:18:40.281931+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:40.281931+0000","last_undegraded":"2024-09-16T22:18:40.281931+0000","last_fullsized":"2024-09-16T22:18:40.281931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2831,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165131+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:41.165131+0000","last_peered":"2024-09-16T22:18:41.165131+0000","last_clean":"2024-09-16T22:18:41.165131+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:41.165131+0000","last_undegraded":"2024-09-16T22:18:41.165131+0000","last_fullsized":"2024-09-16T22:18:41.165131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":328,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249804+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:41.249804+0000","last_peered":"2024-09-16T22:18:41.249804+0000","last_clean":"2024-09-16T22:18:41.249804+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:41.249804+0000","last_undegraded":"2024-09-16T22:18:41.249804+0000","last_fullsized":"2024-09-16T22:18:41.249804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4369,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165125+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:41.165125+0000","last_peered":"2024-09-16T22:18:41.165125+0000","last_clean":"2024-09-16T22:18:41.165125+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:41.165125+0000","last_undegraded":"2024-09-16T22:18:41.165125+0000","last_fullsized":"2024-09-16T22:18:41.165125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3385,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282139+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:40.282139+0000","last_peered":"2024-09-16T22:18:40.282139+0000","last_clean":"2024-09-16T22:18:40.282139+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:40.282139+0000","last_undegraded":"2024-09-16T22:18:40.282139+0000","last_fullsized":"2024-09-16T22:18:40.282139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6647,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249819+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:41.249819+0000","last_peered":"2024-09-16T22:18:41.249819+0000","last_clean":"2024-09-16T22:18:41.249819+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:41.249819+0000","last_undegraded":"2024-09-16T22:18:41.249819+0000","last_fullsized":"2024-09-16T22:18:41.249819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5785,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281952+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:40.281952+0000","last_peered":"2024-09-16T22:18:40.281952+0000","last_clean":"2024-09-16T22:18:40.281952+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:40.281952+0000","last_undegraded":"2024-09-16T22:18:40.281952+0000","last_fullsized":"2024-09-16T22:18:40.281952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12343,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51600,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320240,"statfs":{"total":96636764160,"available":96583925760,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:46.179 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T22:18:46.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:46.477 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:46.488 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T22:18:44.598336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371852,"kb_used_data":15420,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602868,"statfs":{"total":773094113280,"available":772713336832,"internally_reserved":0,"allocated":15790080,"data_stored":8115499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001291"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3054,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288827+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288827+0000","last_peered":"2024-09-16T22:18:41.288827+0000","last_clean":"2024-09-16T22:18:41.288827+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288827+0000","last_undegraded":"2024-09-16T22:18:41.288827+0000","last_fullsized":"2024-09-16T22:18:41.288827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6350,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249768+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:41.249768+0000","last_peered":"2024-09-16T22:18:41.249768+0000","last_clean":"2024-09-16T22:18:41.249768+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:41.249768+0000","last_undegraded":"2024-09-16T22:18:41.249768+0000","last_fullsized":"2024-09-16T22:18:41.249768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7246,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165166+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:41.165166+0000","last_peered":"2024-09-16T22:18:41.165166+0000","last_clean":"2024-09-16T22:18:41.165166+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:41.165166+0000","last_undegraded":"2024-09-16T22:18:41.165166+0000","last_fullsized":"2024-09-16T22:18:41.165166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22402,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249985+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:41.249985+0000","last_peered":"2024-09-16T22:18:41.249985+0000","last_clean":"2024-09-16T22:18:41.249985+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:41.249985+0000","last_undegraded":"2024-09-16T22:18:41.249985+0000","last_fullsized":"2024-09-16T22:18:41.249985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165208+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:41.165208+0000","last_peered":"2024-09-16T22:18:41.165208+0000","last_clean":"2024-09-16T22:18:41.165208+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:41.165208+0000","last_undegraded":"2024-09-16T22:18:41.165208+0000","last_fullsized":"2024-09-16T22:18:41.165208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249841+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:41.249841+0000","last_peered":"2024-09-16T22:18:41.249841+0000","last_clean":"2024-09-16T22:18:41.249841+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:41.249841+0000","last_undegraded":"2024-09-16T22:18:41.249841+0000","last_fullsized":"2024-09-16T22:18:41.249841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2794,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282330+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:40.282330+0000","last_peered":"2024-09-16T22:18:40.282330+0000","last_clean":"2024-09-16T22:18:40.282330+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:40.282330+0000","last_undegraded":"2024-09-16T22:18:40.282330+0000","last_fullsized":"2024-09-16T22:18:40.282330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13679,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165219+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:41.165219+0000","last_peered":"2024-09-16T22:18:41.165219+0000","last_clean":"2024-09-16T22:18:41.165219+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:41.165219+0000","last_undegraded":"2024-09-16T22:18:41.165219+0000","last_fullsized":"2024-09-16T22:18:41.165219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165245+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:41.165245+0000","last_peered":"2024-09-16T22:18:41.165245+0000","last_clean":"2024-09-16T22:18:41.165245+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:41.165245+0000","last_undegraded":"2024-09-16T22:18:41.165245+0000","last_fullsized":"2024-09-16T22:18:41.165245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165239+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:41.165239+0000","last_peered":"2024-09-16T22:18:41.165239+0000","last_clean":"2024-09-16T22:18:41.165239+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:41.165239+0000","last_undegraded":"2024-09-16T22:18:41.165239+0000","last_fullsized":"2024-09-16T22:18:41.165239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2669,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8903,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281931+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:40.281931+0000","last_peered":"2024-09-16T22:18:40.281931+0000","last_clean":"2024-09-16T22:18:40.281931+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:40.281931+0000","last_undegraded":"2024-09-16T22:18:40.281931+0000","last_fullsized":"2024-09-16T22:18:40.281931+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2831,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165131+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:41.165131+0000","last_peered":"2024-09-16T22:18:41.165131+0000","last_clean":"2024-09-16T22:18:41.165131+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:41.165131+0000","last_undegraded":"2024-09-16T22:18:41.165131+0000","last_fullsized":"2024-09-16T22:18:41.165131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":328,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249804+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:41.249804+0000","last_peered":"2024-09-16T22:18:41.249804+0000","last_clean":"2024-09-16T22:18:41.249804+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:41.249804+0000","last_undegraded":"2024-09-16T22:18:41.249804+0000","last_fullsized":"2024-09-16T22:18:41.249804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4369,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.165125+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:41.165125+0000","last_peered":"2024-09-16T22:18:41.165125+0000","last_clean":"2024-09-16T22:18:41.165125+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:41.165125+0000","last_undegraded":"2024-09-16T22:18:41.165125+0000","last_fullsized":"2024-09-16T22:18:41.165125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3385,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.282139+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:40.282139+0000","last_peered":"2024-09-16T22:18:40.282139+0000","last_clean":"2024-09-16T22:18:40.282139+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:40.282139+0000","last_undegraded":"2024-09-16T22:18:40.282139+0000","last_fullsized":"2024-09-16T22:18:40.282139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6647,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249819+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:41.249819+0000","last_peered":"2024-09-16T22:18:41.249819+0000","last_clean":"2024-09-16T22:18:41.249819+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:41.249819+0000","last_undegraded":"2024-09-16T22:18:41.249819+0000","last_fullsized":"2024-09-16T22:18:41.249819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5785,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:40.281952+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:40.281952+0000","last_peered":"2024-09-16T22:18:40.281952+0000","last_clean":"2024-09-16T22:18:40.281952+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:40.281952+0000","last_undegraded":"2024-09-16T22:18:40.281952+0000","last_fullsized":"2024-09-16T22:18:40.281952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12343,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.249707+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:41.249707+0000","last_peered":"2024-09-16T22:18:41.249707+0000","last_clean":"2024-09-16T22:18:41.249707+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:41.249707+0000","last_undegraded":"2024-09-16T22:18:41.249707+0000","last_fullsized":"2024-09-16T22:18:41.249707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509478,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51600,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320240,"statfs":{"total":96636764160,"available":96583925760,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2416640,"data_stored":1455043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":864656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:46.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T22:18:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:46.791 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":153,"fsid":"57fe7da5-e356-48a4-ba33-5558b1026a32","created":"2024-09-16T22:15:44.319460+0000","modified":"2024-09-16T22:18:41.283657+0000","last_up_change":"2024-09-16T22:15:56.374983+0000","last_in_change":"2024-09-16T22:15:46.438697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T22:15:56.583719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T22:16:02.234097+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T22:16:13.789572+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44de8de0-e16a-4325-94b9-1cc61f4829db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6817","nonce":2585124124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6819","nonce":2585124124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6823","nonce":2585124124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2585124124},{"type":"v1","addr":"172.21.15.31:6821","nonce":2585124124}]},"public_addr":"172.21.15.31:6817/2585124124","cluster_addr":"172.21.15.31:6819/2585124124","heartbeat_back_addr":"172.21.15.31:6823/2585124124","heartbeat_front_addr":"172.21.15.31:6821/2585124124","state":["exists","up"]},{"osd":1,"uuid":"0e05fbd1-9707-40b8-91a3-198c6f5b7865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6801","nonce":3271285223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6803","nonce":3271285223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6807","nonce":3271285223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3271285223},{"type":"v1","addr":"172.21.15.31:6805","nonce":3271285223}]},"public_addr":"172.21.15.31:6801/3271285223","cluster_addr":"172.21.15.31:6803/3271285223","heartbeat_back_addr":"172.21.15.31:6807/3271285223","heartbeat_front_addr":"172.21.15.31:6805/3271285223","state":["exists","up"]},{"osd":2,"uuid":"b080bc29-0a92-4200-ae3f-0a17a1722145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6809","nonce":2874892035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6811","nonce":2874892035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6815","nonce":2874892035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2874892035},{"type":"v1","addr":"172.21.15.31:6813","nonce":2874892035}]},"public_addr":"172.21.15.31:6809/2874892035","cluster_addr":"172.21.15.31:6811/2874892035","heartbeat_back_addr":"172.21.15.31:6815/2874892035","heartbeat_front_addr":"172.21.15.31:6813/2874892035","state":["exists","up"]},{"osd":3,"uuid":"0bc64c75-486c-40ac-abaf-381e151623bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6825","nonce":3768411773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6827","nonce":3768411773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6831","nonce":3768411773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3768411773},{"type":"v1","addr":"172.21.15.31:6829","nonce":3768411773}]},"public_addr":"172.21.15.31:6825/3768411773","cluster_addr":"172.21.15.31:6827/3768411773","heartbeat_back_addr":"172.21.15.31:6831/3768411773","heartbeat_front_addr":"172.21.15.31:6829/3768411773","state":["exists","up"]},{"osd":4,"uuid":"fae627d1-6bc2-4e5b-85e1-334e5e05f830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6825","nonce":3421901083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6827","nonce":3421901083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6831","nonce":3421901083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3421901083},{"type":"v1","addr":"172.21.15.92:6829","nonce":3421901083}]},"public_addr":"172.21.15.92:6825/3421901083","cluster_addr":"172.21.15.92:6827/3421901083","heartbeat_back_addr":"172.21.15.92:6831/3421901083","heartbeat_front_addr":"172.21.15.92:6829/3421901083","state":["exists","up"]},{"osd":5,"uuid":"989a79cd-ef84-4bf3-a64f-9d101d0b29d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6817","nonce":335833224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6819","nonce":335833224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6823","nonce":335833224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":335833224},{"type":"v1","addr":"172.21.15.92:6821","nonce":335833224}]},"public_addr":"172.21.15.92:6817/335833224","cluster_addr":"172.21.15.92:6819/335833224","heartbeat_back_addr":"172.21.15.92:6823/335833224","heartbeat_front_addr":"172.21.15.92:6821/335833224","state":["exists","up"]},{"osd":6,"uuid":"92f145b7-e868-43fd-9abe-68c43a407ae2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6809","nonce":1592419018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6811","nonce":1592419018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6815","nonce":1592419018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1592419018},{"type":"v1","addr":"172.21.15.92:6813","nonce":1592419018}]},"public_addr":"172.21.15.92:6809/1592419018","cluster_addr":"172.21.15.92:6811/1592419018","heartbeat_back_addr":"172.21.15.92:6815/1592419018","heartbeat_front_addr":"172.21.15.92:6813/1592419018","state":["exists","up"]},{"osd":7,"uuid":"373f8002-53a4-4ca7-bbbf-70e8d56b3571","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6801","nonce":3448871901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6803","nonce":3448871901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6807","nonce":3448871901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3448871901},{"type":"v1","addr":"172.21.15.92:6805","nonce":3448871901}]},"public_addr":"172.21.15.92:6801/3448871901","cluster_addr":"172.21.15.92:6803/3448871901","heartbeat_back_addr":"172.21.15.92:6807/3448871901","heartbeat_front_addr":"172.21.15.92:6805/3448871901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.033837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.799099+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.769970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.965029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.706036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:53.158188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.722931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T22:15:52.485009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T22:18:47.806 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T22:18:47.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:47.947 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:47.947 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:47.947 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:47.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T22:18:48.257 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T22:18:48.271 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T22:18:48.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:48.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:48.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:48.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T22:18:48.708 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T22:18:48.721 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T22:18:48.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:48.848 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:48.848 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:48.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T22:18:49.178 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T22:18:49.191 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T22:18:49.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:49.324 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:49.324 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:49.324 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:49.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T22:18:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T22:18:49.646 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T22:18:49.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:49.774 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:49.774 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:49.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T22:18:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T22:18:50.109 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T22:18:50.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:50.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T22:18:50.554 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T22:18:50.568 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T22:18:50.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:50.695 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:50.695 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:50.695 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:50.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T22:18:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T22:18:51.041 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T22:18:51.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T22:18:51.169 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T22:18:51.169 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T22:18:51.169 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T22:18:51.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T22:18:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T22:18:51.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:18:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:18:51.791 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:18:51.802 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-16T22:18:50.599391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21348,"ondisk_log_size":21348,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371888,"kb_used_data":15480,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754602832,"statfs":{"total":773094113280,"available":772713299968,"internally_reserved":0,"allocated":15851520,"data_stored":8144564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":364760625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001966"},"pg_stats":[{"pgid":"3.1f","version":"137'772","reported_seq":5803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007789+0000","last_change":"2024-09-16T22:18:25.198245+0000","last_active":"2024-09-16T22:18:42.007789+0000","last_peered":"2024-09-16T22:18:42.007789+0000","last_clean":"2024-09-16T22:18:42.007789+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:42.007789+0000","last_undegraded":"2024-09-16T22:18:42.007789+0000","last_fullsized":"2024-09-16T22:18:42.007789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:18:18.490739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"136'706","reported_seq":5688,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007842+0000","last_change":"2024-09-16T22:18:25.198565+0000","last_active":"2024-09-16T22:18:42.007842+0000","last_peered":"2024-09-16T22:18:42.007842+0000","last_clean":"2024-09-16T22:18:42.007842+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:42.007842+0000","last_undegraded":"2024-09-16T22:18:42.007842+0000","last_fullsized":"2024-09-16T22:18:42.007842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:28.153984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"137'723","reported_seq":3055,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288880+0000","last_change":"2024-09-16T22:18:25.261888+0000","last_active":"2024-09-16T22:18:41.288880+0000","last_peered":"2024-09-16T22:18:41.288880+0000","last_clean":"2024-09-16T22:18:41.288880+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:18:41.288880+0000","last_undegraded":"2024-09-16T22:18:41.288880+0000","last_fullsized":"2024-09-16T22:18:41.288880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:52.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"137'549","reported_seq":6353,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617464+0000","last_change":"2024-09-16T22:18:30.222684+0000","last_active":"2024-09-16T22:18:42.617464+0000","last_peered":"2024-09-16T22:18:42.617464+0000","last_clean":"2024-09-16T22:18:42.617464+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:42.617464+0000","last_undegraded":"2024-09-16T22:18:42.617464+0000","last_fullsized":"2024-09-16T22:18:42.617464+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:13.329691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'616","reported_seq":7249,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614684+0000","last_change":"2024-09-16T22:18:25.218369+0000","last_active":"2024-09-16T22:18:42.614684+0000","last_peered":"2024-09-16T22:18:42.614684+0000","last_clean":"2024-09-16T22:18:42.614684+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:42.614684+0000","last_undegraded":"2024-09-16T22:18:42.614684+0000","last_fullsized":"2024-09-16T22:18:42.614684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:27.891231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"136'638","reported_seq":22405,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.618076+0000","last_change":"2024-09-16T22:18:30.221940+0000","last_active":"2024-09-16T22:18:42.618076+0000","last_peered":"2024-09-16T22:18:42.618076+0000","last_clean":"2024-09-16T22:18:42.618076+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:42.618076+0000","last_undegraded":"2024-09-16T22:18:42.618076+0000","last_fullsized":"2024-09-16T22:18:42.618076+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:06.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'660","reported_seq":11809,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614616+0000","last_change":"2024-09-16T22:18:25.217922+0000","last_active":"2024-09-16T22:18:42.614616+0000","last_peered":"2024-09-16T22:18:42.614616+0000","last_clean":"2024-09-16T22:18:42.614616+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:42.614616+0000","last_undegraded":"2024-09-16T22:18:42.614616+0000","last_fullsized":"2024-09-16T22:18:42.614616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:40.144497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"136'512","reported_seq":5867,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007855+0000","last_change":"2024-09-16T22:18:25.198640+0000","last_active":"2024-09-16T22:18:42.007855+0000","last_peered":"2024-09-16T22:18:42.007855+0000","last_clean":"2024-09-16T22:18:42.007855+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:42.007855+0000","last_undegraded":"2024-09-16T22:18:42.007855+0000","last_fullsized":"2024-09-16T22:18:42.007855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:13.882776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'975","reported_seq":8326,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007799+0000","last_change":"2024-09-16T22:18:25.198305+0000","last_active":"2024-09-16T22:18:42.007799+0000","last_peered":"2024-09-16T22:18:42.007799+0000","last_clean":"2024-09-16T22:18:42.007799+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:42.007799+0000","last_undegraded":"2024-09-16T22:18:42.007799+0000","last_fullsized":"2024-09-16T22:18:42.007799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:39.125426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"135'542","reported_seq":4993,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617013+0000","last_change":"2024-09-16T22:18:25.194030+0000","last_active":"2024-09-16T22:18:42.617013+0000","last_peered":"2024-09-16T22:18:42.617013+0000","last_clean":"2024-09-16T22:18:42.617013+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:42.617013+0000","last_undegraded":"2024-09-16T22:18:42.617013+0000","last_fullsized":"2024-09-16T22:18:42.617013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:27.610168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"135'792","reported_seq":2796,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617839+0000","last_change":"2024-09-16T22:18:25.204227+0000","last_active":"2024-09-16T22:18:42.617839+0000","last_peered":"2024-09-16T22:18:42.617839+0000","last_clean":"2024-09-16T22:18:42.617839+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:42.617839+0000","last_undegraded":"2024-09-16T22:18:42.617839+0000","last_fullsized":"2024-09-16T22:18:42.617839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:30.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'407","reported_seq":10348,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288757+0000","last_change":"2024-09-16T22:18:25.261118+0000","last_active":"2024-09-16T22:18:41.288757+0000","last_peered":"2024-09-16T22:18:41.288757+0000","last_clean":"2024-09-16T22:18:41.288757+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:41.288757+0000","last_undegraded":"2024-09-16T22:18:41.288757+0000","last_fullsized":"2024-09-16T22:18:41.288757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:28.037206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288811+0000","last_change":"2024-09-16T22:16:05.430191+0000","last_active":"2024-09-16T22:18:41.288811+0000","last_peered":"2024-09-16T22:18:41.288811+0000","last_clean":"2024-09-16T22:18:41.288811+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:41.288811+0000","last_undegraded":"2024-09-16T22:18:41.288811+0000","last_fullsized":"2024-09-16T22:18:41.288811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:46.616316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'974","reported_seq":13681,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614705+0000","last_change":"2024-09-16T22:18:25.218747+0000","last_active":"2024-09-16T22:18:42.614705+0000","last_peered":"2024-09-16T22:18:42.614705+0000","last_clean":"2024-09-16T22:18:42.614705+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:42.614705+0000","last_undegraded":"2024-09-16T22:18:42.614705+0000","last_fullsized":"2024-09-16T22:18:42.614705+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:18.009271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":276,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614653+0000","last_change":"2024-09-16T22:16:05.824266+0000","last_active":"2024-09-16T22:18:42.614653+0000","last_peered":"2024-09-16T22:18:42.614653+0000","last_clean":"2024-09-16T22:18:42.614653+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:42.614653+0000","last_undegraded":"2024-09-16T22:18:42.614653+0000","last_fullsized":"2024-09-16T22:18:42.614653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:24.172838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'819","reported_seq":5216,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614688+0000","last_change":"2024-09-16T22:18:25.218650+0000","last_active":"2024-09-16T22:18:42.614688+0000","last_peered":"2024-09-16T22:18:42.614688+0000","last_clean":"2024-09-16T22:18:42.614688+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:42.614688+0000","last_undegraded":"2024-09-16T22:18:42.614688+0000","last_fullsized":"2024-09-16T22:18:42.614688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.041950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288971+0000","last_change":"2024-09-16T22:16:05.430483+0000","last_active":"2024-09-16T22:18:41.288971+0000","last_peered":"2024-09-16T22:18:41.288971+0000","last_clean":"2024-09-16T22:18:41.288971+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:41.288971+0000","last_undegraded":"2024-09-16T22:18:41.288971+0000","last_fullsized":"2024-09-16T22:18:41.288971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:12.548281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'485","reported_seq":13798,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287953+0000","last_change":"2024-09-16T22:18:25.192058+0000","last_active":"2024-09-16T22:18:41.287953+0000","last_peered":"2024-09-16T22:18:41.287953+0000","last_clean":"2024-09-16T22:18:41.287953+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:41.287953+0000","last_undegraded":"2024-09-16T22:18:41.287953+0000","last_fullsized":"2024-09-16T22:18:41.287953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:06.161336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007750+0000","last_change":"2024-09-16T22:16:05.426817+0000","last_active":"2024-09-16T22:18:42.007750+0000","last_peered":"2024-09-16T22:18:42.007750+0000","last_clean":"2024-09-16T22:18:42.007750+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:42.007750+0000","last_undegraded":"2024-09-16T22:18:42.007750+0000","last_fullsized":"2024-09-16T22:18:42.007750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:04.290037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"137'491","reported_seq":2671,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617341+0000","last_change":"2024-09-16T22:18:25.194655+0000","last_active":"2024-09-16T22:18:42.617341+0000","last_peered":"2024-09-16T22:18:42.617341+0000","last_clean":"2024-09-16T22:18:42.617341+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:42.617341+0000","last_undegraded":"2024-09-16T22:18:42.617341+0000","last_fullsized":"2024-09-16T22:18:42.617341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:14.740127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"137'1046","reported_seq":8905,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.618082+0000","last_change":"2024-09-16T22:18:25.204054+0000","last_active":"2024-09-16T22:18:42.618082+0000","last_peered":"2024-09-16T22:18:42.618082+0000","last_clean":"2024-09-16T22:18:42.618082+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:42.618082+0000","last_undegraded":"2024-09-16T22:18:42.618082+0000","last_fullsized":"2024-09-16T22:18:42.618082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:36.222059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":283,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288829+0000","last_change":"2024-09-16T22:16:05.430175+0000","last_active":"2024-09-16T22:18:41.288829+0000","last_peered":"2024-09-16T22:18:41.288829+0000","last_clean":"2024-09-16T22:18:41.288829+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:41.288829+0000","last_undegraded":"2024-09-16T22:18:41.288829+0000","last_fullsized":"2024-09-16T22:18:41.288829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:01.397208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'392","reported_seq":11298,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.007666+0000","last_change":"2024-09-16T22:18:25.197714+0000","last_active":"2024-09-16T22:18:42.007666+0000","last_peered":"2024-09-16T22:18:42.007666+0000","last_clean":"2024-09-16T22:18:42.007666+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:42.007666+0000","last_undegraded":"2024-09-16T22:18:42.007666+0000","last_fullsized":"2024-09-16T22:18:42.007666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:19.180748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314278+0000","last_change":"2024-09-16T22:16:05.930514+0000","last_active":"2024-09-16T22:18:41.314278+0000","last_peered":"2024-09-16T22:18:41.314278+0000","last_clean":"2024-09-16T22:18:41.314278+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:41.314278+0000","last_undegraded":"2024-09-16T22:18:41.314278+0000","last_fullsized":"2024-09-16T22:18:41.314278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:59.824550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"137'614","reported_seq":16274,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287910+0000","last_change":"2024-09-16T22:18:25.192065+0000","last_active":"2024-09-16T22:18:41.287910+0000","last_peered":"2024-09-16T22:18:41.287910+0000","last_clean":"2024-09-16T22:18:41.287910+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:41.287910+0000","last_undegraded":"2024-09-16T22:18:41.287910+0000","last_fullsized":"2024-09-16T22:18:41.287910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:04.673792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288595+0000","last_change":"2024-09-16T22:16:05.429704+0000","last_active":"2024-09-16T22:18:41.288595+0000","last_peered":"2024-09-16T22:18:41.288595+0000","last_clean":"2024-09-16T22:18:41.288595+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:41.288595+0000","last_undegraded":"2024-09-16T22:18:41.288595+0000","last_fullsized":"2024-09-16T22:18:41.288595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:39.594341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"136'358","reported_seq":2833,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614643+0000","last_change":"2024-09-16T22:18:25.217918+0000","last_active":"2024-09-16T22:18:42.614643+0000","last_peered":"2024-09-16T22:18:42.614643+0000","last_clean":"2024-09-16T22:18:42.614643+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:42.614643+0000","last_undegraded":"2024-09-16T22:18:42.614643+0000","last_fullsized":"2024-09-16T22:18:42.614643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:32.588129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":282,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288866+0000","last_change":"2024-09-16T22:16:05.430542+0000","last_active":"2024-09-16T22:18:41.288866+0000","last_peered":"2024-09-16T22:18:41.288866+0000","last_clean":"2024-09-16T22:18:41.288866+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:41.288866+0000","last_undegraded":"2024-09-16T22:18:41.288866+0000","last_fullsized":"2024-09-16T22:18:41.288866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:02.402201+0000","last_clean_scrub_stamp":"2024-09-16T22:16:02.402201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:38.913640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":330,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617722+0000","last_change":"2024-09-16T22:18:30.222152+0000","last_active":"2024-09-16T22:18:42.617722+0000","last_peered":"2024-09-16T22:18:42.617722+0000","last_clean":"2024-09-16T22:18:42.617722+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:42.617722+0000","last_undegraded":"2024-09-16T22:18:42.617722+0000","last_fullsized":"2024-09-16T22:18:42.617722+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:15:57.377279+0000","last_clean_scrub_stamp":"2024-09-16T22:15:57.377279+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:43.889103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'1108","reported_seq":4371,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.614616+0000","last_change":"2024-09-16T22:18:25.218376+0000","last_active":"2024-09-16T22:18:42.614616+0000","last_peered":"2024-09-16T22:18:42.614616+0000","last_clean":"2024-09-16T22:18:42.614616+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:42.614616+0000","last_undegraded":"2024-09-16T22:18:42.614616+0000","last_fullsized":"2024-09-16T22:18:42.614616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:00.821928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"137'539","reported_seq":5241,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288643+0000","last_change":"2024-09-16T22:18:25.261551+0000","last_active":"2024-09-16T22:18:41.288643+0000","last_peered":"2024-09-16T22:18:41.288643+0000","last_clean":"2024-09-16T22:18:41.288643+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:41.288643+0000","last_undegraded":"2024-09-16T22:18:41.288643+0000","last_fullsized":"2024-09-16T22:18:41.288643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:41.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"136'705","reported_seq":2784,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288688+0000","last_change":"2024-09-16T22:18:25.261773+0000","last_active":"2024-09-16T22:18:41.288688+0000","last_peered":"2024-09-16T22:18:41.288688+0000","last_clean":"2024-09-16T22:18:41.288688+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:41.288688+0000","last_undegraded":"2024-09-16T22:18:41.288688+0000","last_fullsized":"2024-09-16T22:18:41.288688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:59.866759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'484","reported_seq":14692,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.314272+0000","last_change":"2024-09-16T22:18:25.275025+0000","last_active":"2024-09-16T22:18:41.314272+0000","last_peered":"2024-09-16T22:18:41.314272+0000","last_clean":"2024-09-16T22:18:41.314272+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:41.314272+0000","last_undegraded":"2024-09-16T22:18:41.314272+0000","last_fullsized":"2024-09-16T22:18:41.314272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:18.879016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"133'456","reported_seq":3387,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617668+0000","last_change":"2024-09-16T22:18:25.203459+0000","last_active":"2024-09-16T22:18:42.617668+0000","last_peered":"2024-09-16T22:18:42.617668+0000","last_clean":"2024-09-16T22:18:42.617668+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:42.617668+0000","last_undegraded":"2024-09-16T22:18:42.617668+0000","last_fullsized":"2024-09-16T22:18:42.617668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:24.735561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"135'779","reported_seq":3365,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288839+0000","last_change":"2024-09-16T22:18:25.262247+0000","last_active":"2024-09-16T22:18:41.288839+0000","last_peered":"2024-09-16T22:18:41.288839+0000","last_clean":"2024-09-16T22:18:41.288839+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:41.288839+0000","last_undegraded":"2024-09-16T22:18:41.288839+0000","last_fullsized":"2024-09-16T22:18:41.288839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:42.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"136'600","reported_seq":6649,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617114+0000","last_change":"2024-09-16T22:18:25.194366+0000","last_active":"2024-09-16T22:18:42.617114+0000","last_peered":"2024-09-16T22:18:42.617114+0000","last_clean":"2024-09-16T22:18:42.617114+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:42.617114+0000","last_undegraded":"2024-09-16T22:18:42.617114+0000","last_fullsized":"2024-09-16T22:18:42.617114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:36.020406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'827","reported_seq":5787,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617432+0000","last_change":"2024-09-16T22:18:25.204497+0000","last_active":"2024-09-16T22:18:42.617432+0000","last_peered":"2024-09-16T22:18:42.617432+0000","last_clean":"2024-09-16T22:18:42.617432+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:42.617432+0000","last_undegraded":"2024-09-16T22:18:42.617432+0000","last_fullsized":"2024-09-16T22:18:42.617432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:57.568335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"133'443","reported_seq":12345,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:42.617626+0000","last_change":"2024-09-16T22:18:30.222744+0000","last_active":"2024-09-16T22:18:42.617626+0000","last_peered":"2024-09-16T22:18:42.617626+0000","last_clean":"2024-09-16T22:18:42.617626+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:42.617626+0000","last_undegraded":"2024-09-16T22:18:42.617626+0000","last_fullsized":"2024-09-16T22:18:42.617626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:17.416340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"137'1296","reported_seq":5217,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288580+0000","last_change":"2024-09-16T22:18:25.261372+0000","last_active":"2024-09-16T22:18:41.288580+0000","last_peered":"2024-09-16T22:18:41.288580+0000","last_clean":"2024-09-16T22:18:41.288580+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:41.288580+0000","last_undegraded":"2024-09-16T22:18:41.288580+0000","last_fullsized":"2024-09-16T22:18:41.288580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:26.661601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"137'439","reported_seq":2727,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.287915+0000","last_change":"2024-09-16T22:18:25.192452+0000","last_active":"2024-09-16T22:18:41.287915+0000","last_peered":"2024-09-16T22:18:41.287915+0000","last_clean":"2024-09-16T22:18:41.287915+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:41.287915+0000","last_undegraded":"2024-09-16T22:18:41.287915+0000","last_fullsized":"2024-09-16T22:18:41.287915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:00.704977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"136'526","reported_seq":3642,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:41.288627+0000","last_change":"2024-09-16T22:18:25.260416+0000","last_active":"2024-09-16T22:18:41.288627+0000","last_peered":"2024-09-16T22:18:41.288627+0000","last_clean":"2024-09-16T22:18:41.288627+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:41.288627+0000","last_undegraded":"2024-09-16T22:18:41.288627+0000","last_fullsized":"2024-09-16T22:18:41.288627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:16:14.110173+0000","last_clean_scrub_stamp":"2024-09-16T22:16:14.110173+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:08.760268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21273,"ondisk_log_size":21273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51608,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320232,"statfs":{"total":96636764160,"available":96583917568,"internally_reserved":0,"allocated":2428928,"data_stored":1460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51024,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320816,"statfs":{"total":96636764160,"available":96584515584,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55184,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316656,"statfs":{"total":96636764160,"available":96580255744,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37720,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334120,"statfs":{"total":96636764160,"available":96598138880,"internally_reserved":0,"allocated":2428928,"data_stored":1460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55184,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316656,"statfs":{"total":96636764160,"available":96580255744,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28824,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343016,"statfs":{"total":96636764160,"available":96607248384,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:18:51.807 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.807 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.808 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.809 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T22:16:02.402201+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=2, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T22:15:57.377279+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=15, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.810 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.811 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.812 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.812 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T22:16:14.110173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=16, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=22, tm_min=18, tm_sec=46, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T22:18:51.812 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T22:19:11.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T22:19:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T22:19:12.125 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T22:19:12.136 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-16T22:19:10.604175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":1654,"num_read":266387,"num_read_kb":913965,"num_write":22256,"num_write_kb":590103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23008,"ondisk_log_size":23008,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380152,"kb_used_data":15480,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754594568,"statfs":{"total":773094113280,"available":772704837632,"internally_reserved":0,"allocated":15851520,"data_stored":8144564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":373280305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002879"},"pg_stats":[{"pgid":"3.1f","version":"153'825","reported_seq":5866,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:55.716483+0000","last_change":"2024-09-16T22:18:55.716483+0000","last_active":"2024-09-16T22:18:55.716483+0000","last_peered":"2024-09-16T22:18:55.716483+0000","last_clean":"2024-09-16T22:18:55.716483+0000","last_became_active":"2024-09-16T22:16:15.129307+0000","last_became_peered":"2024-09-16T22:16:15.129307+0000","last_unstale":"2024-09-16T22:18:55.716483+0000","last_undegraded":"2024-09-16T22:18:55.716483+0000","last_fullsized":"2024-09-16T22:18:55.716483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'825","last_scrub_stamp":"2024-09-16T22:18:55.716384+0000","last_deep_scrub":"153'825","last_deep_scrub_stamp":"2024-09-16T22:18:55.716384+0000","last_clean_scrub_stamp":"2024-09-16T22:18:55.716384+0000","objects_scrubbed":53,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:19.786635+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0.0012601940000000001,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":7646,"num_read_kb":45733,"num_write":892,"num_write_kb":6342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"153'757","reported_seq":5749,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:53.631371+0000","last_change":"2024-09-16T22:18:53.631371+0000","last_active":"2024-09-16T22:18:53.631371+0000","last_peered":"2024-09-16T22:18:53.631371+0000","last_clean":"2024-09-16T22:18:53.631371+0000","last_became_active":"2024-09-16T22:16:15.132353+0000","last_became_peered":"2024-09-16T22:16:15.132353+0000","last_unstale":"2024-09-16T22:18:53.631371+0000","last_undegraded":"2024-09-16T22:18:53.631371+0000","last_fullsized":"2024-09-16T22:18:53.631371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'757","last_scrub_stamp":"2024-09-16T22:18:53.631298+0000","last_deep_scrub":"153'757","last_deep_scrub_stamp":"2024-09-16T22:18:53.631298+0000","last_clean_scrub_stamp":"2024-09-16T22:18:53.631298+0000","objects_scrubbed":51,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:28.128185+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.001046574,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7629,"num_read_kb":27424,"num_write":902,"num_write_kb":35898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"153'757","reported_seq":3098,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:19:00.843075+0000","last_change":"2024-09-16T22:19:00.843075+0000","last_active":"2024-09-16T22:19:00.843075+0000","last_peered":"2024-09-16T22:19:00.843075+0000","last_clean":"2024-09-16T22:19:00.843075+0000","last_became_active":"2024-09-16T22:16:15.136478+0000","last_became_peered":"2024-09-16T22:16:15.136478+0000","last_unstale":"2024-09-16T22:19:00.843075+0000","last_undegraded":"2024-09-16T22:19:00.843075+0000","last_fullsized":"2024-09-16T22:19:00.843075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'757","last_scrub_stamp":"2024-09-16T22:19:00.842971+0000","last_deep_scrub":"153'757","last_deep_scrub_stamp":"2024-09-16T22:19:00.842971+0000","last_clean_scrub_stamp":"2024-09-16T22:19:00.842971+0000","objects_scrubbed":34,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:36.989205+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.0024675410000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":4050,"num_read_kb":26859,"num_write":917,"num_write_kb":11244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"153'599","reported_seq":6413,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.056130+0000","last_change":"2024-09-16T22:18:52.056130+0000","last_active":"2024-09-16T22:18:52.056130+0000","last_peered":"2024-09-16T22:18:52.056130+0000","last_clean":"2024-09-16T22:18:52.056130+0000","last_became_active":"2024-09-16T22:18:30.221176+0000","last_became_peered":"2024-09-16T22:18:30.221176+0000","last_unstale":"2024-09-16T22:18:52.056130+0000","last_undegraded":"2024-09-16T22:18:52.056130+0000","last_fullsized":"2024-09-16T22:18:52.056130+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'599","last_scrub_stamp":"2024-09-16T22:18:52.056063+0000","last_deep_scrub":"153'599","last_deep_scrub_stamp":"2024-09-16T22:18:52.056063+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.056063+0000","objects_scrubbed":51,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:33.329865+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.00082742799999999995,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":6530,"num_read_kb":15225,"num_write":501,"num_write_kb":37530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"153'654","reported_seq":7296,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:55.775717+0000","last_change":"2024-09-16T22:18:55.775717+0000","last_active":"2024-09-16T22:18:55.775717+0000","last_peered":"2024-09-16T22:18:55.775717+0000","last_clean":"2024-09-16T22:18:55.775717+0000","last_became_active":"2024-09-16T22:16:15.135446+0000","last_became_peered":"2024-09-16T22:16:15.135446+0000","last_unstale":"2024-09-16T22:18:55.775717+0000","last_undegraded":"2024-09-16T22:18:55.775717+0000","last_fullsized":"2024-09-16T22:18:55.775717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'654","last_scrub_stamp":"2024-09-16T22:18:55.775614+0000","last_deep_scrub":"153'654","last_deep_scrub_stamp":"2024-09-16T22:18:55.775614+0000","last_clean_scrub_stamp":"2024-09-16T22:18:55.775614+0000","objects_scrubbed":38,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:31.669388+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.00085576200000000004,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7841,"num_read_kb":21333,"num_write":569,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"153'690","reported_seq":22467,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:48.972517+0000","last_change":"2024-09-16T22:18:48.972517+0000","last_active":"2024-09-16T22:18:48.972517+0000","last_peered":"2024-09-16T22:18:48.972517+0000","last_clean":"2024-09-16T22:18:48.972517+0000","last_became_active":"2024-09-16T22:18:30.220264+0000","last_became_peered":"2024-09-16T22:18:30.220264+0000","last_unstale":"2024-09-16T22:18:48.972517+0000","last_undegraded":"2024-09-16T22:18:48.972517+0000","last_fullsized":"2024-09-16T22:18:48.972517+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'690","last_scrub_stamp":"2024-09-16T22:18:48.972446+0000","last_deep_scrub":"153'690","last_deep_scrub_stamp":"2024-09-16T22:18:48.972446+0000","last_clean_scrub_stamp":"2024-09-16T22:18:48.972446+0000","objects_scrubbed":52,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:17:03.659110+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.001590129,"stat_sum":{"num_bytes":19,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":23684,"num_read_kb":53027,"num_write":538,"num_write_kb":17321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"153'710","reported_seq":11869,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:56.806567+0000","last_change":"2024-09-16T22:18:56.806567+0000","last_active":"2024-09-16T22:18:56.806567+0000","last_peered":"2024-09-16T22:18:56.806567+0000","last_clean":"2024-09-16T22:18:56.806567+0000","last_became_active":"2024-09-16T22:16:15.133962+0000","last_became_peered":"2024-09-16T22:16:15.133962+0000","last_unstale":"2024-09-16T22:18:56.806567+0000","last_undegraded":"2024-09-16T22:18:56.806567+0000","last_fullsized":"2024-09-16T22:18:56.806567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'710","last_scrub_stamp":"2024-09-16T22:18:56.806489+0000","last_deep_scrub":"153'710","last_deep_scrub_stamp":"2024-09-16T22:18:56.806489+0000","last_clean_scrub_stamp":"2024-09-16T22:18:56.806489+0000","objects_scrubbed":50,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:38.333280+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0.00050208200000000005,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":14116,"num_read_kb":34576,"num_write":561,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"153'569","reported_seq":5934,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:54.681925+0000","last_change":"2024-09-16T22:18:54.681925+0000","last_active":"2024-09-16T22:18:54.681925+0000","last_peered":"2024-09-16T22:18:54.681925+0000","last_clean":"2024-09-16T22:18:54.681925+0000","last_became_active":"2024-09-16T22:16:15.128534+0000","last_became_peered":"2024-09-16T22:16:15.128534+0000","last_unstale":"2024-09-16T22:18:54.681925+0000","last_undegraded":"2024-09-16T22:18:54.681925+0000","last_fullsized":"2024-09-16T22:18:54.681925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'569","last_scrub_stamp":"2024-09-16T22:18:54.681837+0000","last_deep_scrub":"153'569","last_deep_scrub_stamp":"2024-09-16T22:18:54.681837+0000","last_clean_scrub_stamp":"2024-09-16T22:18:54.681837+0000","objects_scrubbed":57,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:18.587235+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0.0014329220000000001,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":5548,"num_read_kb":15058,"num_write":499,"num_write_kb":8931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"153'1026","reported_seq":8387,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.664324+0000","last_change":"2024-09-16T22:18:52.664324+0000","last_active":"2024-09-16T22:18:52.664324+0000","last_peered":"2024-09-16T22:18:52.664324+0000","last_clean":"2024-09-16T22:18:52.664324+0000","last_became_active":"2024-09-16T22:16:15.129729+0000","last_became_peered":"2024-09-16T22:16:15.129729+0000","last_unstale":"2024-09-16T22:18:52.664324+0000","last_undegraded":"2024-09-16T22:18:52.664324+0000","last_fullsized":"2024-09-16T22:18:52.664324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'1026","last_scrub_stamp":"2024-09-16T22:18:52.664248+0000","last_deep_scrub":"153'1026","last_deep_scrub_stamp":"2024-09-16T22:18:52.664248+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.664248+0000","objects_scrubbed":51,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:19.499937+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0.001305823,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":9870,"num_read_kb":17567,"num_write":1303,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"153'586","reported_seq":5046,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:49.223621+0000","last_change":"2024-09-16T22:18:49.223621+0000","last_active":"2024-09-16T22:18:49.223621+0000","last_peered":"2024-09-16T22:18:49.223621+0000","last_clean":"2024-09-16T22:18:49.223621+0000","last_became_active":"2024-09-16T22:16:15.130939+0000","last_became_peered":"2024-09-16T22:16:15.130939+0000","last_unstale":"2024-09-16T22:18:49.223621+0000","last_undegraded":"2024-09-16T22:18:49.223621+0000","last_fullsized":"2024-09-16T22:18:49.223621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'586","last_scrub_stamp":"2024-09-16T22:18:49.223521+0000","last_deep_scrub":"153'586","last_deep_scrub_stamp":"2024-09-16T22:18:49.223521+0000","last_clean_scrub_stamp":"2024-09-16T22:18:49.223521+0000","objects_scrubbed":44,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:51.430263+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0.00056098300000000003,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":4955,"num_read_kb":21732,"num_write":564,"num_write_kb":35353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"153'840","reported_seq":2853,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.046451+0000","last_change":"2024-09-16T22:18:52.046451+0000","last_active":"2024-09-16T22:18:52.046451+0000","last_peered":"2024-09-16T22:18:52.046451+0000","last_clean":"2024-09-16T22:18:52.046451+0000","last_became_active":"2024-09-16T22:16:15.133708+0000","last_became_peered":"2024-09-16T22:16:15.133708+0000","last_unstale":"2024-09-16T22:18:52.046451+0000","last_undegraded":"2024-09-16T22:18:52.046451+0000","last_fullsized":"2024-09-16T22:18:52.046451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'840","last_scrub_stamp":"2024-09-16T22:18:52.046354+0000","last_deep_scrub":"153'840","last_deep_scrub_stamp":"2024-09-16T22:18:52.046354+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.046354+0000","objects_scrubbed":48,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:34.313453+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0013205510000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3643,"num_read_kb":14355,"num_write":888,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"153'460","reported_seq":10411,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:59.897664+0000","last_change":"2024-09-16T22:18:59.897664+0000","last_active":"2024-09-16T22:18:59.897664+0000","last_peered":"2024-09-16T22:18:59.897664+0000","last_clean":"2024-09-16T22:18:59.897664+0000","last_became_active":"2024-09-16T22:16:15.131525+0000","last_became_peered":"2024-09-16T22:16:15.131525+0000","last_unstale":"2024-09-16T22:18:59.897664+0000","last_undegraded":"2024-09-16T22:18:59.897664+0000","last_fullsized":"2024-09-16T22:18:59.897664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'460","last_scrub_stamp":"2024-09-16T22:18:59.897585+0000","last_deep_scrub":"153'460","last_deep_scrub_stamp":"2024-09-16T22:18:59.897585+0000","last_clean_scrub_stamp":"2024-09-16T22:18:59.897585+0000","objects_scrubbed":53,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:05.628300+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0017182129999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":9686,"num_read_kb":39717,"num_write":331,"num_write_kb":14219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":289,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:53.846235+0000","last_change":"2024-09-16T22:18:53.846235+0000","last_active":"2024-09-16T22:18:53.846235+0000","last_peered":"2024-09-16T22:18:53.846235+0000","last_clean":"2024-09-16T22:18:53.846235+0000","last_became_active":"2024-09-16T22:16:04.049324+0000","last_became_peered":"2024-09-16T22:16:04.049324+0000","last_unstale":"2024-09-16T22:18:53.846235+0000","last_undegraded":"2024-09-16T22:18:53.846235+0000","last_fullsized":"2024-09-16T22:18:53.846235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:53.846149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:53.846149+0000","last_clean_scrub_stamp":"2024-09-16T22:18:53.846149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:19.893104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010037290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"153'1018","reported_seq":13734,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.723771+0000","last_change":"2024-09-16T22:18:52.723771+0000","last_active":"2024-09-16T22:18:52.723771+0000","last_peered":"2024-09-16T22:18:52.723771+0000","last_clean":"2024-09-16T22:18:52.723771+0000","last_became_active":"2024-09-16T22:16:15.129369+0000","last_became_peered":"2024-09-16T22:16:15.129369+0000","last_unstale":"2024-09-16T22:18:52.723771+0000","last_undegraded":"2024-09-16T22:18:52.723771+0000","last_fullsized":"2024-09-16T22:18:52.723771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'1018","last_scrub_stamp":"2024-09-16T22:18:52.723661+0000","last_deep_scrub":"153'1018","last_deep_scrub_stamp":"2024-09-16T22:18:52.723661+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.723661+0000","objects_scrubbed":44,"log_size":1018,"log_dups_size":0,"ondisk_log_size":1018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:34.217767+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.0012961540000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16657,"num_read_kb":34253,"num_write":960,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":284,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.735850+0000","last_change":"2024-09-16T22:18:51.735850+0000","last_active":"2024-09-16T22:18:51.735850+0000","last_peered":"2024-09-16T22:18:51.735850+0000","last_clean":"2024-09-16T22:18:51.735850+0000","last_became_active":"2024-09-16T22:16:04.049601+0000","last_became_peered":"2024-09-16T22:16:04.049601+0000","last_unstale":"2024-09-16T22:18:51.735850+0000","last_undegraded":"2024-09-16T22:18:51.735850+0000","last_fullsized":"2024-09-16T22:18:51.735850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:51.735769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:51.735769+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.735769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:07.412652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051889000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"153'878","reported_seq":5285,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:54.800780+0000","last_change":"2024-09-16T22:18:54.800780+0000","last_active":"2024-09-16T22:18:54.800780+0000","last_peered":"2024-09-16T22:18:54.800780+0000","last_clean":"2024-09-16T22:18:54.800780+0000","last_became_active":"2024-09-16T22:16:15.127623+0000","last_became_peered":"2024-09-16T22:16:15.127623+0000","last_unstale":"2024-09-16T22:18:54.800780+0000","last_undegraded":"2024-09-16T22:18:54.800780+0000","last_fullsized":"2024-09-16T22:18:54.800780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'878","last_scrub_stamp":"2024-09-16T22:18:54.800660+0000","last_deep_scrub":"153'878","last_deep_scrub_stamp":"2024-09-16T22:18:54.800660+0000","last_clean_scrub_stamp":"2024-09-16T22:18:54.800660+0000","objects_scrubbed":59,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:52.704311+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0.001209644,"stat_sum":{"num_bytes":0,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":59,"num_read":6595,"num_read_kb":18130,"num_write":969,"num_write_kb":25301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.819268+0000","last_change":"2024-09-16T22:18:52.819268+0000","last_active":"2024-09-16T22:18:52.819268+0000","last_peered":"2024-09-16T22:18:52.819268+0000","last_clean":"2024-09-16T22:18:52.819268+0000","last_became_active":"2024-09-16T22:16:03.431601+0000","last_became_peered":"2024-09-16T22:16:03.431601+0000","last_unstale":"2024-09-16T22:18:52.819268+0000","last_undegraded":"2024-09-16T22:18:52.819268+0000","last_fullsized":"2024-09-16T22:18:52.819268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:52.819194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:52.819194+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.819194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:50.117400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001297879,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"153'538","reported_seq":13861,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.692340+0000","last_change":"2024-09-16T22:18:51.692340+0000","last_active":"2024-09-16T22:18:51.692340+0000","last_peered":"2024-09-16T22:18:51.692340+0000","last_clean":"2024-09-16T22:18:51.692340+0000","last_became_active":"2024-09-16T22:16:15.127375+0000","last_became_peered":"2024-09-16T22:16:15.127375+0000","last_unstale":"2024-09-16T22:18:51.692340+0000","last_undegraded":"2024-09-16T22:18:51.692340+0000","last_fullsized":"2024-09-16T22:18:51.692340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'538","last_scrub_stamp":"2024-09-16T22:18:51.692256+0000","last_deep_scrub":"153'538","last_deep_scrub_stamp":"2024-09-16T22:18:51.692256+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.692256+0000","objects_scrubbed":53,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:24.994470+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0.000499104,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":13217,"num_read_kb":25139,"num_write":464,"num_write_kb":19453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:50.646696+0000","last_change":"2024-09-16T22:18:50.646696+0000","last_active":"2024-09-16T22:18:50.646696+0000","last_peered":"2024-09-16T22:18:50.646696+0000","last_clean":"2024-09-16T22:18:50.646696+0000","last_became_active":"2024-09-16T22:16:03.705082+0000","last_became_peered":"2024-09-16T22:16:03.705082+0000","last_unstale":"2024-09-16T22:18:50.646696+0000","last_undegraded":"2024-09-16T22:18:50.646696+0000","last_fullsized":"2024-09-16T22:18:50.646696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:50.646611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:50.646611+0000","last_clean_scrub_stamp":"2024-09-16T22:18:50.646611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:06.350573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040749500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"153'540","reported_seq":2729,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.205723+0000","last_change":"2024-09-16T22:18:51.205723+0000","last_active":"2024-09-16T22:18:51.205723+0000","last_peered":"2024-09-16T22:18:51.205723+0000","last_clean":"2024-09-16T22:18:51.205723+0000","last_became_active":"2024-09-16T22:16:15.129033+0000","last_became_peered":"2024-09-16T22:16:15.129033+0000","last_unstale":"2024-09-16T22:18:51.205723+0000","last_undegraded":"2024-09-16T22:18:51.205723+0000","last_fullsized":"2024-09-16T22:18:51.205723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'540","last_scrub_stamp":"2024-09-16T22:18:51.205609+0000","last_deep_scrub":"153'540","last_deep_scrub_stamp":"2024-09-16T22:18:51.205609+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.205609+0000","objects_scrubbed":49,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:32.225519+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0012471520000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":2600,"num_read_kb":24149,"num_write":465,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"153'1111","reported_seq":8980,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:49.066017+0000","last_change":"2024-09-16T22:18:49.066017+0000","last_active":"2024-09-16T22:18:49.066017+0000","last_peered":"2024-09-16T22:18:49.066017+0000","last_clean":"2024-09-16T22:18:49.066017+0000","last_became_active":"2024-09-16T22:16:15.132701+0000","last_became_peered":"2024-09-16T22:16:15.132701+0000","last_unstale":"2024-09-16T22:18:49.066017+0000","last_undegraded":"2024-09-16T22:18:49.066017+0000","last_fullsized":"2024-09-16T22:18:49.066017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'1111","last_scrub_stamp":"2024-09-16T22:18:49.065912+0000","last_deep_scrub":"153'1111","last_deep_scrub_stamp":"2024-09-16T22:18:49.065912+0000","last_clean_scrub_stamp":"2024-09-16T22:18:49.065912+0000","objects_scrubbed":66,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:37.617079+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0.001111197,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":65,"num_read":10144,"num_read_kb":23404,"num_write":1036,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"153'3","reported_seq":292,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:50.851834+0000","last_change":"2024-09-16T22:18:50.851834+0000","last_active":"2024-09-16T22:18:50.851834+0000","last_peered":"2024-09-16T22:18:50.851834+0000","last_clean":"2024-09-16T22:18:50.851834+0000","last_became_active":"2024-09-16T22:16:03.705241+0000","last_became_peered":"2024-09-16T22:16:03.705241+0000","last_unstale":"2024-09-16T22:18:50.851834+0000","last_undegraded":"2024-09-16T22:18:50.851834+0000","last_fullsized":"2024-09-16T22:18:50.851834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'3","last_scrub_stamp":"2024-09-16T22:18:50.851749+0000","last_deep_scrub":"153'3","last_deep_scrub_stamp":"2024-09-16T22:18:50.851749+0000","last_clean_scrub_stamp":"2024-09-16T22:18:50.851749+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:08.521494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00099855400000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"153'435","reported_seq":11350,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.687862+0000","last_change":"2024-09-16T22:18:51.687862+0000","last_active":"2024-09-16T22:18:51.687862+0000","last_peered":"2024-09-16T22:18:51.687862+0000","last_clean":"2024-09-16T22:18:51.687862+0000","last_became_active":"2024-09-16T22:17:10.126732+0000","last_became_peered":"2024-09-16T22:17:10.126732+0000","last_unstale":"2024-09-16T22:18:51.687862+0000","last_undegraded":"2024-09-16T22:18:51.687862+0000","last_fullsized":"2024-09-16T22:18:51.687862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'435","last_scrub_stamp":"2024-09-16T22:18:51.687685+0000","last_deep_scrub":"153'435","last_deep_scrub_stamp":"2024-09-16T22:18:51.687685+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.687685+0000","objects_scrubbed":43,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:21.912838+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.00075683800000000002,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11128,"num_read_kb":63835,"num_write":305,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":7569502,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":287,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.019396+0000","last_change":"2024-09-16T22:18:52.019396+0000","last_active":"2024-09-16T22:18:52.019396+0000","last_peered":"2024-09-16T22:18:52.019396+0000","last_clean":"2024-09-16T22:18:52.019396+0000","last_became_active":"2024-09-16T22:16:03.431566+0000","last_became_peered":"2024-09-16T22:16:03.431566+0000","last_unstale":"2024-09-16T22:18:52.019396+0000","last_undegraded":"2024-09-16T22:18:52.019396+0000","last_fullsized":"2024-09-16T22:18:52.019396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:52.019310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:52.019310+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.019310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:37.881571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036158400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"153'658","reported_seq":16327,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:53.588278+0000","last_change":"2024-09-16T22:18:53.588278+0000","last_active":"2024-09-16T22:18:53.588278+0000","last_peered":"2024-09-16T22:18:53.588278+0000","last_clean":"2024-09-16T22:18:53.588278+0000","last_became_active":"2024-09-16T22:16:46.282123+0000","last_became_peered":"2024-09-16T22:16:46.282123+0000","last_unstale":"2024-09-16T22:18:53.588278+0000","last_undegraded":"2024-09-16T22:18:53.588278+0000","last_fullsized":"2024-09-16T22:18:53.588278+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'658","last_scrub_stamp":"2024-09-16T22:18:53.588169+0000","last_deep_scrub":"153'658","last_deep_scrub_stamp":"2024-09-16T22:18:53.588169+0000","last_clean_scrub_stamp":"2024-09-16T22:18:53.588169+0000","objects_scrubbed":44,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:32.174900+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00050567299999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":16141,"num_read_kb":102655,"num_write":701,"num_write_kb":44012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":244664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":289,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:49.885990+0000","last_change":"2024-09-16T22:18:49.885990+0000","last_active":"2024-09-16T22:18:49.885990+0000","last_peered":"2024-09-16T22:18:49.885990+0000","last_clean":"2024-09-16T22:18:49.885990+0000","last_became_active":"2024-09-16T22:16:03.431531+0000","last_became_peered":"2024-09-16T22:16:03.431531+0000","last_unstale":"2024-09-16T22:18:49.885990+0000","last_undegraded":"2024-09-16T22:18:49.885990+0000","last_fullsized":"2024-09-16T22:18:49.885990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T22:18:49.885871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T22:18:49.885871+0000","last_clean_scrub_stamp":"2024-09-16T22:18:49.885871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:00.953281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054532099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"153'411","reported_seq":2896,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:50.759206+0000","last_change":"2024-09-16T22:18:50.759206+0000","last_active":"2024-09-16T22:18:50.759206+0000","last_peered":"2024-09-16T22:18:50.759206+0000","last_clean":"2024-09-16T22:18:50.759206+0000","last_became_active":"2024-09-16T22:16:15.131196+0000","last_became_peered":"2024-09-16T22:16:15.131196+0000","last_unstale":"2024-09-16T22:18:50.759206+0000","last_undegraded":"2024-09-16T22:18:50.759206+0000","last_fullsized":"2024-09-16T22:18:50.759206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'411","last_scrub_stamp":"2024-09-16T22:18:50.759128+0000","last_deep_scrub":"153'411","last_deep_scrub_stamp":"2024-09-16T22:18:50.759128+0000","last_clean_scrub_stamp":"2024-09-16T22:18:50.759128+0000","objects_scrubbed":53,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:43.065027+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00049570199999999999,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2135,"num_read_kb":24542,"num_write":342,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"153'2","reported_seq":291,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.826827+0000","last_change":"2024-09-16T22:18:51.826827+0000","last_active":"2024-09-16T22:18:51.826827+0000","last_peered":"2024-09-16T22:18:51.826827+0000","last_clean":"2024-09-16T22:18:51.826827+0000","last_became_active":"2024-09-16T22:16:03.422611+0000","last_became_peered":"2024-09-16T22:16:03.422611+0000","last_unstale":"2024-09-16T22:18:51.826827+0000","last_undegraded":"2024-09-16T22:18:51.826827+0000","last_fullsized":"2024-09-16T22:18:51.826827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'2","last_scrub_stamp":"2024-09-16T22:18:51.826758+0000","last_deep_scrub":"153'2","last_deep_scrub_stamp":"2024-09-16T22:18:51.826758+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.826758+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:03.728355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013567780000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"153'74","reported_seq":340,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:49.929556+0000","last_change":"2024-09-16T22:18:49.929556+0000","last_active":"2024-09-16T22:18:49.929556+0000","last_peered":"2024-09-16T22:18:49.929556+0000","last_clean":"2024-09-16T22:18:49.929556+0000","last_became_active":"2024-09-16T22:18:30.220945+0000","last_became_peered":"2024-09-16T22:18:30.220945+0000","last_unstale":"2024-09-16T22:18:49.929556+0000","last_undegraded":"2024-09-16T22:18:49.929556+0000","last_fullsized":"2024-09-16T22:18:49.929556+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'74","last_scrub_stamp":"2024-09-16T22:18:49.929467+0000","last_deep_scrub":"153'74","last_deep_scrub_stamp":"2024-09-16T22:18:49.929467+0000","last_clean_scrub_stamp":"2024-09-16T22:18:49.929467+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:06.582952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"153'1164","reported_seq":4437,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:53.771721+0000","last_change":"2024-09-16T22:18:53.771721+0000","last_active":"2024-09-16T22:18:53.771721+0000","last_peered":"2024-09-16T22:18:53.771721+0000","last_clean":"2024-09-16T22:18:53.771721+0000","last_became_active":"2024-09-16T22:16:15.127241+0000","last_became_peered":"2024-09-16T22:16:15.127241+0000","last_unstale":"2024-09-16T22:18:53.771721+0000","last_undegraded":"2024-09-16T22:18:53.771721+0000","last_fullsized":"2024-09-16T22:18:53.771721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'1164","last_scrub_stamp":"2024-09-16T22:18:53.771622+0000","last_deep_scrub":"153'1164","last_deep_scrub_stamp":"2024-09-16T22:18:53.771622+0000","last_clean_scrub_stamp":"2024-09-16T22:18:53.771622+0000","objects_scrubbed":56,"log_size":1164,"log_dups_size":0,"ondisk_log_size":1164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:35.231394+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0.00093674600000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":7717,"num_read_kb":19731,"num_write":1343,"num_write_kb":26561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"153'591","reported_seq":5303,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:54.872511+0000","last_change":"2024-09-16T22:18:54.872511+0000","last_active":"2024-09-16T22:18:54.872511+0000","last_peered":"2024-09-16T22:18:54.872511+0000","last_clean":"2024-09-16T22:18:54.872511+0000","last_became_active":"2024-09-16T22:16:15.129708+0000","last_became_peered":"2024-09-16T22:16:15.129708+0000","last_unstale":"2024-09-16T22:18:54.872511+0000","last_undegraded":"2024-09-16T22:18:54.872511+0000","last_fullsized":"2024-09-16T22:18:54.872511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'591","last_scrub_stamp":"2024-09-16T22:18:54.872428+0000","last_deep_scrub":"153'591","last_deep_scrub_stamp":"2024-09-16T22:18:54.872428+0000","last_clean_scrub_stamp":"2024-09-16T22:18:54.872428+0000","objects_scrubbed":52,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:06.088195+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.0021295060000000002,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5637,"num_read_kb":19275,"num_write":542,"num_write_kb":6622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"153'772","reported_seq":2861,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:56.902762+0000","last_change":"2024-09-16T22:18:56.902762+0000","last_active":"2024-09-16T22:18:56.902762+0000","last_peered":"2024-09-16T22:18:56.902762+0000","last_clean":"2024-09-16T22:18:56.902762+0000","last_became_active":"2024-09-16T22:16:15.125151+0000","last_became_peered":"2024-09-16T22:16:15.125151+0000","last_unstale":"2024-09-16T22:18:56.902762+0000","last_undegraded":"2024-09-16T22:18:56.902762+0000","last_fullsized":"2024-09-16T22:18:56.902762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'772","last_scrub_stamp":"2024-09-16T22:18:56.902675+0000","last_deep_scrub":"153'772","last_deep_scrub_stamp":"2024-09-16T22:18:56.902675+0000","last_clean_scrub_stamp":"2024-09-16T22:18:56.902675+0000","objects_scrubbed":67,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:55.930494+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0023483200000000001,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":66,"num_read":3924,"num_read_kb":7721,"num_write":677,"num_write_kb":3311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"153'542","reported_seq":14760,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:53.075812+0000","last_change":"2024-09-16T22:18:53.075812+0000","last_active":"2024-09-16T22:18:53.075812+0000","last_peered":"2024-09-16T22:18:53.075812+0000","last_clean":"2024-09-16T22:18:53.075812+0000","last_became_active":"2024-09-16T22:16:15.127718+0000","last_became_peered":"2024-09-16T22:16:15.127718+0000","last_unstale":"2024-09-16T22:18:53.075812+0000","last_undegraded":"2024-09-16T22:18:53.075812+0000","last_fullsized":"2024-09-16T22:18:53.075812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'542","last_scrub_stamp":"2024-09-16T22:18:53.075726+0000","last_deep_scrub":"153'542","last_deep_scrub_stamp":"2024-09-16T22:18:53.075726+0000","last_clean_scrub_stamp":"2024-09-16T22:18:53.075726+0000","objects_scrubbed":58,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:07.431211+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0.00060673499999999996,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":14605,"num_read_kb":47479,"num_write":395,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"153'511","reported_seq":3452,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:50.060245+0000","last_change":"2024-09-16T22:18:50.060245+0000","last_active":"2024-09-16T22:18:50.060245+0000","last_peered":"2024-09-16T22:18:50.060245+0000","last_clean":"2024-09-16T22:18:50.060245+0000","last_became_active":"2024-09-16T22:16:15.132365+0000","last_became_peered":"2024-09-16T22:16:15.132365+0000","last_unstale":"2024-09-16T22:18:50.060245+0000","last_undegraded":"2024-09-16T22:18:50.060245+0000","last_fullsized":"2024-09-16T22:18:50.060245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'511","last_scrub_stamp":"2024-09-16T22:18:50.060161+0000","last_deep_scrub":"153'511","last_deep_scrub_stamp":"2024-09-16T22:18:50.060161+0000","last_clean_scrub_stamp":"2024-09-16T22:18:50.060161+0000","objects_scrubbed":55,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:04.357665+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0.00060350200000000001,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":55,"num_read":2759,"num_read_kb":16330,"num_write":366,"num_write_kb":8505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"153'831","reported_seq":3427,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:55.897783+0000","last_change":"2024-09-16T22:18:55.897783+0000","last_active":"2024-09-16T22:18:55.897783+0000","last_peered":"2024-09-16T22:18:55.897783+0000","last_clean":"2024-09-16T22:18:55.897783+0000","last_became_active":"2024-09-16T22:16:15.128554+0000","last_became_peered":"2024-09-16T22:16:15.128554+0000","last_unstale":"2024-09-16T22:18:55.897783+0000","last_undegraded":"2024-09-16T22:18:55.897783+0000","last_fullsized":"2024-09-16T22:18:55.897783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'831","last_scrub_stamp":"2024-09-16T22:18:55.897697+0000","last_deep_scrub":"153'831","last_deep_scrub_stamp":"2024-09-16T22:18:55.897697+0000","last_clean_scrub_stamp":"2024-09-16T22:18:55.897697+0000","objects_scrubbed":52,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:33.845506+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0.002697049,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":5834,"num_read_kb":8733,"num_write":795,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"153'650","reported_seq":6709,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:50.216345+0000","last_change":"2024-09-16T22:18:50.216345+0000","last_active":"2024-09-16T22:18:50.216345+0000","last_peered":"2024-09-16T22:18:50.216345+0000","last_clean":"2024-09-16T22:18:50.216345+0000","last_became_active":"2024-09-16T22:16:15.135587+0000","last_became_peered":"2024-09-16T22:16:15.135587+0000","last_unstale":"2024-09-16T22:18:50.216345+0000","last_undegraded":"2024-09-16T22:18:50.216345+0000","last_fullsized":"2024-09-16T22:18:50.216345+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'650","last_scrub_stamp":"2024-09-16T22:18:50.216282+0000","last_deep_scrub":"153'650","last_deep_scrub_stamp":"2024-09-16T22:18:50.216282+0000","last_clean_scrub_stamp":"2024-09-16T22:18:50.216282+0000","objects_scrubbed":50,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:58.788598+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.000858514,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":6777,"num_read_kb":60010,"num_write":597,"num_write_kb":36651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"153'881","reported_seq":5851,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.027751+0000","last_change":"2024-09-16T22:18:51.027751+0000","last_active":"2024-09-16T22:18:51.027751+0000","last_peered":"2024-09-16T22:18:51.027751+0000","last_clean":"2024-09-16T22:18:51.027751+0000","last_became_active":"2024-09-16T22:16:15.137901+0000","last_became_peered":"2024-09-16T22:16:15.137901+0000","last_unstale":"2024-09-16T22:18:51.027751+0000","last_undegraded":"2024-09-16T22:18:51.027751+0000","last_fullsized":"2024-09-16T22:18:51.027751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'881","last_scrub_stamp":"2024-09-16T22:18:51.027678+0000","last_deep_scrub":"153'881","last_deep_scrub_stamp":"2024-09-16T22:18:51.027678+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.027678+0000","objects_scrubbed":54,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:27.080877+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0015114289999999999,"stat_sum":{"num_bytes":0,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":54,"num_read":7653,"num_read_kb":9569,"num_write":898,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"153'495","reported_seq":12407,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:51.005970+0000","last_change":"2024-09-16T22:18:51.005970+0000","last_active":"2024-09-16T22:18:51.005970+0000","last_peered":"2024-09-16T22:18:51.005970+0000","last_clean":"2024-09-16T22:18:51.005970+0000","last_became_active":"2024-09-16T22:18:30.221405+0000","last_became_peered":"2024-09-16T22:18:30.221405+0000","last_unstale":"2024-09-16T22:18:51.005970+0000","last_undegraded":"2024-09-16T22:18:51.005970+0000","last_fullsized":"2024-09-16T22:18:51.005970+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'495","last_scrub_stamp":"2024-09-16T22:18:51.005892+0000","last_deep_scrub":"153'495","last_deep_scrub_stamp":"2024-09-16T22:18:51.005892+0000","last_clean_scrub_stamp":"2024-09-16T22:18:51.005892+0000","objects_scrubbed":52,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:26.101384+0000","scrub_duration":35,"objects_trimmed":2,"snaptrim_duration":0.00070396200000000003,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":11888,"num_read_kb":29946,"num_write":398,"num_write_kb":8007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"153'1348","reported_seq":5279,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:57.880184+0000","last_change":"2024-09-16T22:18:57.880184+0000","last_active":"2024-09-16T22:18:57.880184+0000","last_peered":"2024-09-16T22:18:57.880184+0000","last_clean":"2024-09-16T22:18:57.880184+0000","last_became_active":"2024-09-16T22:16:15.129664+0000","last_became_peered":"2024-09-16T22:16:15.129664+0000","last_unstale":"2024-09-16T22:18:57.880184+0000","last_undegraded":"2024-09-16T22:18:57.880184+0000","last_fullsized":"2024-09-16T22:18:57.880184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'1348","last_scrub_stamp":"2024-09-16T22:18:57.880106+0000","last_deep_scrub":"153'1348","last_deep_scrub_stamp":"2024-09-16T22:18:57.880106+0000","last_clean_scrub_stamp":"2024-09-16T22:18:57.880106+0000","objects_scrubbed":52,"log_size":1348,"log_dups_size":0,"ondisk_log_size":1348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:28:09.714274+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0.0019982239999999998,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":9036,"num_read_kb":22288,"num_write":1613,"num_write_kb":8908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"153'505","reported_seq":2803,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:52.654133+0000","last_change":"2024-09-16T22:18:52.654133+0000","last_active":"2024-09-16T22:18:52.654133+0000","last_peered":"2024-09-16T22:18:52.654133+0000","last_clean":"2024-09-16T22:18:52.654133+0000","last_became_active":"2024-09-16T22:16:15.130852+0000","last_became_peered":"2024-09-16T22:16:15.130852+0000","last_unstale":"2024-09-16T22:18:52.654133+0000","last_undegraded":"2024-09-16T22:18:52.654133+0000","last_fullsized":"2024-09-16T22:18:52.654133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'505","last_scrub_stamp":"2024-09-16T22:18:52.654036+0000","last_deep_scrub":"153'505","last_deep_scrub_stamp":"2024-09-16T22:18:52.654036+0000","last_clean_scrub_stamp":"2024-09-16T22:18:52.654036+0000","objects_scrubbed":66,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:06.333113+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0.00090034999999999996,"stat_sum":{"num_bytes":0,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":66,"num_read":2399,"num_read_kb":17983,"num_write":337,"num_write_kb":6867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"153'577","reported_seq":3703,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T22:18:58.891845+0000","last_change":"2024-09-16T22:18:58.891845+0000","last_active":"2024-09-16T22:18:58.891845+0000","last_peered":"2024-09-16T22:18:58.891845+0000","last_clean":"2024-09-16T22:18:58.891845+0000","last_became_active":"2024-09-16T22:16:15.135439+0000","last_became_peered":"2024-09-16T22:16:15.135439+0000","last_unstale":"2024-09-16T22:18:58.891845+0000","last_undegraded":"2024-09-16T22:18:58.891845+0000","last_fullsized":"2024-09-16T22:18:58.891845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'577","last_scrub_stamp":"2024-09-16T22:18:58.891764+0000","last_deep_scrub":"153'577","last_deep_scrub_stamp":"2024-09-16T22:18:58.891764+0000","last_clean_scrub_stamp":"2024-09-16T22:18:58.891764+0000","objects_scrubbed":51,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:42:05.059181+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0.0010411190000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":4277,"num_read_kb":6132,"num_write":485,"num_write_kb":6346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1658,"num_object_clones":0,"num_object_copies":3316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1658,"num_whiteouts":1654,"num_read":266321,"num_read_kb":913910,"num_write":22153,"num_write_kb":588473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":7814166,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22929,"ondisk_log_size":22929,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51604,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320236,"statfs":{"total":96636764160,"available":96583921664,"internally_reserved":0,"allocated":2428928,"data_stored":1460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1835008,"data_stored":870488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37716,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334124,"statfs":{"total":96636764160,"available":96598142976,"internally_reserved":0,"allocated":2428928,"data_stored":1460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28812,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343028,"statfs":{"total":96636764160,"available":96607260672,"internally_reserved":0,"allocated":1830912,"data_stored":870469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T22:19:12.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T22:19:12.455 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T22:19:12.455 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T22:19:12.455 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T22:19:12.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.493 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T22:19:12.493 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T22:19:12.493 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.566 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T22:19:12.567 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T22:19:12.567 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.636 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T22:19:12.637 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T22:19:12.637 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.708 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T22:19:12.708 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T22:19:12.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.749 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T22:19:12.750 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T22:19:12.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.795 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T22:19:12.795 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T22:19:12.795 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.835 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T22:19:12.835 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T22:19:12.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.903 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T22:19:12.903 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T22:19:12.903 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T22:19:12.904 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.930 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T22:19:12.930 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T22:19:12.930 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.975 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T22:19:12.975 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T22:19:12.975 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T22:19:12.975 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:12.986 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T22:19:12.986 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T22:19:12.986 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:13.038 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T22:19:13.038 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T22:19:13.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T22:19:13.051 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T22:19:13.051 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T22:19:13.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-16T22:19:13.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:19:13.116 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T22:19:13.348 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:19:13.348 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T22:19:13.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:19:13.477 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T22:19:13.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:19:13.545 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T22:19:13.603 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:19:13.603 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T22:19:13.765 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:19:13.765 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T22:19:13.853 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:19:13.853 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T22:19:13.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:19:13.911 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T22:19:13.986 INFO:tasks.ceph:Archiving mon data... 2024-09-16T22:19:13.987 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/data/mon.a.tgz 2024-09-16T22:19:13.987 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-16T22:19:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.OqRzuFDCkc 2024-09-16T22:19:14.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.OqRzuFDCkc 2024-09-16T22:19:14.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.OqRzuFDCkc 2024-09-16T22:19:14.246 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.OqRzuFDCkc is 380KB 2024-09-16T22:19:14.343 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.OqRzuFDCkc 2024-09-16T22:19:14.359 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/data/mon.c.tgz 2024-09-16T22:19:14.359 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-16T22:19:14.414 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.K5v0ZCUvff 2024-09-16T22:19:14.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.K5v0ZCUvff 2024-09-16T22:19:14.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.K5v0ZCUvff 2024-09-16T22:19:14.669 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.K5v0ZCUvff is 513KB 2024-09-16T22:19:14.767 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.K5v0ZCUvff 2024-09-16T22:19:14.782 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/data/mon.b.tgz 2024-09-16T22:19:14.783 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-16T22:19:14.801 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.e0d1IyRqUQ 2024-09-16T22:19:14.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.e0d1IyRqUQ 2024-09-16T22:19:14.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.e0d1IyRqUQ 2024-09-16T22:19:15.073 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.e0d1IyRqUQ is 514KB 2024-09-16T22:19:15.171 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.e0d1IyRqUQ 2024-09-16T22:19:15.186 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T22:19:15.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T22:19:15.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T22:19:15.271 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T22:19:15.306 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T22:19:15.306 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T22:19:15.307 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi031/crash 2024-09-16T22:19:15.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T22:19:15.338 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi092/crash 2024-09-16T22:19:15.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T22:19:15.370 INFO:tasks.ceph:Compressing logs... 2024-09-16T22:19:15.370 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T22:19:15.383 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T22:19:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T22:19:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58311.log 2024-09-16T22:19:15.453 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32983.log 2024-09-16T22:19:15.453 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T22:19:15.453 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.32983.log: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T22:19:15.453 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32983.log.gz 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32943.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32943.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33063.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32943.log.gz 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33234.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33063.log.gz 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33194.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33234.log: /var/log/ceph/ceph-osd.6.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33234.log.gz 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33023.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33194.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33194.log.gz 2024-09-16T22:19:15.454 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33023.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33023.log.gz 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33103.log 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-mon.b.log 2024-09-16T22:19:15.455 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33103.log.gz 2024-09-16T22:19:15.514 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.b.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43401.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.58311.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.58311.log.gz 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.57227.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43401.log.gz 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56295.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57227.log.gz 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58101.log 2024-09-16T22:19:15.529 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56295.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39322.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58101.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39322.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42385.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52224.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45127.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42385.log.gz 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-09-16T22:19:15.530 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52224.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57568.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45127.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58983.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58689.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57568.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57567.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58983.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51546.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58689.log.gz 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40213.log 2024-09-16T22:19:15.531 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57567.log.gz 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43739.log 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51546.log.gz 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40213.log.gz 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48543.log 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43739.log.gz 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2024-09-16T22:19:15.532 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40315.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48543.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44297.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58941.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40315.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44297.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40981.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58941.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41406.log 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz 2024-09-16T22:19:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58815.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40981.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53361.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41406.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58815.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51372.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53361.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54059.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40309.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51372.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54059.log.gz 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-09-16T22:19:15.534 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40309.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41244.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57825.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57765.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57825.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.57825.log.gz /var/log/ceph/ceph-client.admin.55821.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57231.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57765.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55821.log.gz 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51426.log 2024-09-16T22:19:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57231.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51426.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58563.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56589.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 0.0% 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58563.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57225.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56589.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58731.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-09-16T22:19:15.536 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57225.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48564.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58731.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51169.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48564.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51169.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44339.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46790.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44339.log.gz 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48005.log 2024-09-16T22:19:15.537 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46790.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40897.log 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48005.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.40897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56799.log 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56399.log 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.0.40897.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50730.log 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56799.log.gz 2024-09-16T22:19:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39059.log 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56399.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59025.log 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50730.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40781.log 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39059.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43059.log 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59025.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40781.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.46748.log -- replaced with /var/log/ceph/ceph-client.admin.40781.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42205.log 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43059.log.gz 2024-09-16T22:19:15.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57487.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46748.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42205.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56883.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57487.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56883.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44851.log 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2024-09-16T22:19:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50990.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57824.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44851.log.gz 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58395.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50990.log.gz 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57824.log.gz 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44559.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58395.log.gz 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-09-16T22:19:15.541 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39315.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44559.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39315.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57226.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50771.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57226.log.gz 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39319.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39143.log 2024-09-16T22:19:15.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50771.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40739.log 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39319.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39143.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47629.log 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39857.log 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40739.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40739.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58437.log 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47629.log.gz 2024-09-16T22:19:15.543 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51885.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39857.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58437.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56066.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51885.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57228.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57904.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56066.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57228.log.gz 2024-09-16T22:19:15.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56205.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57904.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56631.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45410.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56205.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58479.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56631.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44421.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45410.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54555.log 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58479.log.gz 2024-09-16T22:19:15.545 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53835.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44421.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39998.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54555.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58647.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53835.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46494.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39998.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58647.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48385.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46494.log.gz 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51249.log 2024-09-16T22:19:15.546 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48385.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54813.log 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: 25.6% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39659.log 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51249.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54813.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39657.log 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39659.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56547.log 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46706.log 0.0% 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39657.log.gz 2024-09-16T22:19:15.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56379.log 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56547.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46706.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39101.log -- replaced with /var/log/ceph/ceph-client.admin.46706.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59109.log 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56379.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59109.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46967.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.48859.log -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-16T22:19:15.548 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48047.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57907.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48859.log.gz 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57907.log.gz 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56337.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39658.log 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2024-09-16T22:19:15.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50472.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56337.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39658.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39658.log.gz -5 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52006.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50472.log: -- /var/log/ceph/ceph-client.admin.40843.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50472.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49517.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52006.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39323.log 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49517.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-16T22:19:15.550 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50052.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39323.log.gz 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56967.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50052.log.gz 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51466.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56967.log.gz 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56715.log 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2024-09-16T22:19:15.551 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58521.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58857.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56715.log.gz 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39660.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58521.log.gz 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58773.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58857.log.gz 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39660.log.gz 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52804.log 2024-09-16T22:19:15.552 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58773.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39560.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52804.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39560.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39555.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39560.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T22:19:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58227.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.58227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46579.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58227.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.46579.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57822.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46579.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38762.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49975.log 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57822.log.gz 2024-09-16T22:19:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46452.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49975.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43124.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56673.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46452.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40375.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56673.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.43640.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57230.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40375.log.gz 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-09-16T22:19:15.555 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57230.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57230.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58143.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40529.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58143.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58143.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40529.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2024-09-16T22:19:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49736.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.46537.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46537.log: /var/log/ceph/ceph-client.admin.59149.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46537.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52960.log: /var/log/ceph/ceph-client.admin.40193.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59149.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57482.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40193.log: /var/log/ceph/ceph-client.admin.57565.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40193.log.gz 2024-09-16T22:19:15.557 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45593.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57482.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40982.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.45593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45635.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45593.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40982.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58353.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43060.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45635.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58353.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57009.log 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43060.log.gz 2024-09-16T22:19:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57009.log.gz 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57232.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57906.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57233.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57232.log.gz 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57906.log.gz 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39321.log 2024-09-16T22:19:15.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57233.log: /var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39321.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58899.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39321.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49260.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58899.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58899.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph.tmp-client.admin.27490.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.27490.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27490.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45837.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45266.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46664.log 2024-09-16T22:19:15.560 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45837.log.gz 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41455.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47129.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46664.log.gz 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39661.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41455.log.gz 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57569.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47129.log.gz 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33628.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39661.log: /var/log/ceph/ceph-client.admin.57569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45067.log 2024-09-16T22:19:15.561 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57569.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39661.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33628.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45067.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45067.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58605.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33326.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45264.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58605.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38966.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.33326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33326.log.gz 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45264.log: /var/log/ceph/ceph-client.admin.40613.log 2024-09-16T22:19:15.562 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45264.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38966.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56482.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40613.log: /var/log/ceph/ceph-client.admin.56757.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40613.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56482.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40571.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56482.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40697.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56757.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40571.log: -- /var/log/ceph/ceph-client.admin.45471.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40571.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40697.log.gz 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56841.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45471.log: /var/log/ceph/ceph-client.admin.52265.log 2024-09-16T22:19:15.563 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56841.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56252.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52265.log: -- /var/log/ceph/ceph-client.admin.45675.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56252.log: 27.5% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55768.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.45675.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.54993.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55768.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53220.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54993.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54993.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40655.log 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53220.log.gz 2024-09-16T22:19:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40254.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40655.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40655.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.43462.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40254.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57051.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56483.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42644.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57051.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57489.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56483.log: /var/log/ceph/ceph-client.admin.42644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39917.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42644.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56483.log.gz 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57489.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54257.log 2024-09-16T22:19:15.565 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57489.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59067.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39917.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51329.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54257.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53459.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59067.log: /var/log/ceph/ceph-client.admin.51329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55170.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59067.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51329.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53459.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51526.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55170.log: /var/log/ceph/ceph-client.admin.51289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54931.log 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55170.log.gz 2024-09-16T22:19:15.566 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51526.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41805.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58185.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54931.log: /var/log/ceph/ceph-client.admin.41805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54931.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41805.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58185.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39995.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48284.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44803.log: /var/log/ceph/ceph-client.admin.39995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40255.log 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39995.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48284.log.gz 2024-09-16T22:19:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47246.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58269.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40255.log: /var/log/ceph/ceph-client.admin.47246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40255.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47246.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45551.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58269.log: /var/log/ceph/ceph-client.admin.43901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58269.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46016.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45551.log: /var/log/ceph/ceph-client.admin.46016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45551.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-09-16T22:19:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39916.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46315.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43580.log: /var/log/ceph/ceph-client.admin.39916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39916.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39915.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46315.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39999.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39915.log: 0.0%/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54216.log 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39915.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39999.log.gz 2024-09-16T22:19:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57486.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54216.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53043.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54216.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56925.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57486.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53043.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42084.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38885.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56925.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56164.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42084.log: /var/log/ceph/ceph-client.admin.38885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42084.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38885.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51209.log 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56164.log.gz 2024-09-16T22:19:15.570 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52611.log 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41647.log 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52611.log: /var/log/ceph/ceph-client.admin.51209.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52611.log.gz 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51209.log.gz 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53875.log 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53875.log: /var/log/ceph/ceph-client.admin.41647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41647.log.gz 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53875.log.gz 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-09-16T22:19:15.571 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T22:19:15.572 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T22:19:15.772 INFO:teuthology.orchestra.run.smithi031.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T22:19:16.900 INFO:teuthology.orchestra.run.smithi092.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T22:19:17.247 INFO:teuthology.orchestra.run.smithi031.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T22:19:17.880 INFO:teuthology.orchestra.run.smithi031.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T22:19:18.467 INFO:teuthology.orchestra.run.smithi092.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T22:19:19.041 INFO:teuthology.orchestra.run.smithi092.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T22:19:19.267 INFO:teuthology.orchestra.run.smithi031.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T22:19:19.284 INFO:teuthology.orchestra.run.smithi031.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T22:19:19.286 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T22:19:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m3.890s 2024-09-16T22:19:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m11.996s 2024-09-16T22:19:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.556s 2024-09-16T22:19:19.619 INFO:teuthology.orchestra.run.smithi092.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T22:19:19.621 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-16T22:19:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m4.193s 2024-09-16T22:19:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m11.960s 2024-09-16T22:19:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.511s 2024-09-16T22:19:19.621 INFO:tasks.ceph:Archiving logs... 2024-09-16T22:19:19.621 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi031/log 2024-09-16T22:19:19.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T22:19:21.229 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi092/log 2024-09-16T22:19:21.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T22:19:22.694 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T22:19:22.703 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T22:19:22.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T22:19:22.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T22:19:22.782 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T22:19:22.790 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T22:19:22.791 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T22:19:22.793 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T22:19:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T22:19:22.810 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-16T22:19:22.814 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T22:19:22.814 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T22:19:22.814 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +3694us[+3687us] +/- 55ms 2024-09-16T22:19:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -2499us[-2506us] +/- 73ms 2024-09-16T22:19:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -3191us[-3197us] +/- 87ms 2024-09-16T22:19:22.815 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +3688us[+3681us] +/- 55ms 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -2495us[-2502us] +/- 73ms 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 -3172us[-3172us] +/- 88ms 2024-09-16T22:19:22.816 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T22:19:22.817 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T22:19:22.830 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T22:19:22.830 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T22:19:22.930 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T22:19:23.016 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T22:19:23.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T22:19:23.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:19:23.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T22:19:23.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:19:23.153 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T22:19:23.162 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T22:19:23.205 INFO:teuthology.task.internal:Duration was 628.670868 seconds 2024-09-16T22:19:23.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T22:19:23.289 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T22:19:23.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T22:19:23.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T22:19:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T22:19:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T22:19:23.770 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T22:19:23.771 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T22:19:23.771 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T22:19:23.797 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-16T22:19:23.797 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T22:19:23.833 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T22:19:23.833 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T22:19:23.840 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T22:19:23.902 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T22:19:23.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T22:19:23.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T22:19:24.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T22:19:24.067 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T22:19:24.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T22:19:24.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T22:19:24.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T22:19:24.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T22:19:24.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T22:19:24.158 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-16T22:19:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-16T22:19:24.202 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T22:19:24.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:19:24.246 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T22:19:24.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T22:19:24.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T22:19:24.271 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T22:19:24.272 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi031 2024-09-16T22:19:24.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T22:19:24.331 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-16_19:45:14-rbd-wip-dis-testing-distro-default-smithi/7907596/remote/smithi092 2024-09-16T22:19:24.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T22:19:24.365 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T22:19:24.365 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T22:19:24.368 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T22:19:24.415 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T22:19:24.428 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T22:19:24.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T22:19:24.457 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T22:19:24.458 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T22:19:24.460 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T22:19:24.473 INFO:teuthology.orchestra.run.smithi031.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 22:19 /home/ubuntu/cephtest 2024-09-16T22:19:24.475 INFO:teuthology.orchestra.run.smithi092.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 22:19 /home/ubuntu/cephtest 2024-09-16T22:19:24.476 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T22:19:24.504 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T22:19:24.692 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 628.6708681583405 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-16T22:19:24.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T22:19:24.774 INFO:teuthology.run:pass